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Foreword 


Distributed  interactive  simulation  (DIS)  is  the  linking  of  aircraft,  tank, 
and  other  military  simulators  in  diverse  locations  so  that  the  crew  of 
one  simulator  can  “see,”  operate  with,  “shoot”  at,  or  be  “destroyed”  by 
the  other  vehicles  being  simulated.  Command  structures  can  also  be 
simulated.  This  allows  forces  to  practice  and  train  in  situations  too  costly  or 
risky  to  practice  with  real  weapons. 

The  simulators  are  technological  descendants  of  the  Link  Trainer,  the  fa¬ 
mous  flight  simulator  introduced  in  1929.  Modem  vehicle  simulators  use  elec¬ 
tronic  digital  computers  to  calculate  how  instrument  indications,  visual  dis¬ 
plays,  and  sounds  should  change  in  response  to  a  user’s  handling  of  controls.  A 
prominent  trend,  over  the  last  decade,  has  been  the  increase  in  the  detail  and 
apparent  realism  with  which  increasingly  affordable  computing  power  can 
generate  and  display  a  scene  that  a  helicopter  pilot  might  see  through  his 
canopy  or  a  tank  crewman  might  see  through  his  periscope. 

The  linking  of  the  simulators’  computers  into  a  network,  using  the  technolo¬ 
gies  and  standard  communications  procedures  used  in  the  Internet,  as  well  as 
others,  allows  each  simulator  crew  to  practice  teamwork  with  other  crews 
This  background  paper  is  the  third  publication  of  the  Office  of  Technology 
Assessment’s  (OTA’s)  assessment  of  combat  modeling  and  simulation 
technologies.  It  complements  OTAs  background  paper.  Virtual  Reality  and 
Technologies  of  Combat  Simulation,  which  focuses  on  the  human-computer 
interface  technologies  used  in  simulations.  The  assessment  was  requested  by 
the  House  Committee  on  Armed  Services  (now  the  House  Committee  on  Na¬ 
tional  Security);  the  Senate  Committee  on  Armed  Services;  and  its  Subcom¬ 
mittee  on  Defense  Technology,  Acquisition,  and  Industrial  Base  (now  the  Sub¬ 
committee  on  Acquisition  and  Technology). 

In  undertaking  this  assessment,  OTA  sought  the  contributions  of  a  wide 
spectrum  of  knowledgeable  individuals  and  organizations.  OTA  gratefully  ac¬ 
knowledges  their  contributions  of  time  and  intellectual  effort.  OTA  also  ap¬ 
preciates  the  help  and  cooperation  of  officials  of  the  Department  of  Defense 
and  the  Department  of  Energy.  As  with  all  OTA  publications,  the  content  of 
this  background  paper  is  the  sole  responsibility  of  the  Office  of  Technology  As¬ 
sessment  and  does  not  necessarily  represent  the  views  of  our  advisors  or  re¬ 
viewers. 
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INTRODUCTION:  PREPARING  FOR  THE  NEXT  WAR 

[The  Romans]  do  not  begin  to  use  their  weapons  first  in  time 
of  war,  nor  do  they  then  put  their  hands  first  into  motion,  while 
they  avoided  so  to  do  in  times  of  peace;  but  as  if  their  weapons  did 
always  cling  to  them,  they  have  never  any  truce  from  warlike  ex¬ 
ercises;  . . .  nor  would  he  be  mistaken  that  should  call  those  their 
exercises  unbloody  battles,  and  their  battles  bloody  exercises. — 
Ravius  Josephus,  De  Bello  Judaic o  [The  Jewish  War]  (79) 

The  military  is  proverbially  accused  of  always  training  for  the 
last  war.  To  avoid  this  predicament,  the  U.S.  military  makes  ex¬ 
tensive  use  of  simulators,  simulations,  and  exercises,  designed  to 
emulate  present  or  projected  conditions.  Models  and  simulations 
are  used  for  several  important  purposes:  training  (to  maintain 
readiness),  analysis  (of  the  effects  of  proposed  tactics  or  system 
acquisitions),  planning  and  rehearsal  of  operations,  and  demon¬ 
stration  of  new  technologies.  Simulators,  such  as  the  Link  Train¬ 
er,  have  been  used  primarily  for  training  and  mission  rehearsal. 
More  abstract  simulations  and  models  have  been  used  for  analy¬ 
sis  and  operations  planning. 

The  simulation  entry  on  the  Critical  Technologies  List  issued 
by  the  Director  of  Defense  Research  and  Engineering  (DDR&E) 
always  stood  as  an  area  of  undisputed  superiority  over  the  former 
Soviet  Union.  In  today’s  changed  world,  modeling  and  simula¬ 
tion  technology  is  one  of  20  “technology  areas”  selected  by  the 
Department  of  Defense  (DoD)  for  research  and  development 
funding  emphasis.  DoD’s  Defense  Modeling  and  Simulation  Of¬ 
fice  (DMSO),  founded  in  1991 ,  has  the  lead  in  structuring  DoD’s 
approach  to  modeling  and  simulation,  especially  the  high-tech 
forms. 
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SOURCE:  Perceptronics,  Inc.,  1995. 


Advances  in  the  technologies  of  microelectron¬ 
ics,  computer  networking,  and  computer  graphics 
have  led  to  supercomputers,  the  Internet,  and  syn¬ 
thetic  environments  for  virtual  reality  (VR).  They 
have  also  made  possible  a  new  kind  of  military 
training,  pioneered  by  the  Defense  Advanced  Re¬ 
search  Projects  Agency  (DARPA — now  ARPA') 
in  its  Simulation  Network  (SIMNET)  program, 
which  began  in  1983. 

SIMNET  began  as  a  Tank  Team  Gunnery 
Trainer  to  replace  existing  tank  simulators. 
Through  adroit  use  of  increasingly  capable  and 
economical  computer  equipment,  SIMNET ’s  de¬ 
velopers  expanded  their  system  from  a  tank  simu¬ 
lator  to  a  tank  battle  simulator  for  company-sized 
units.  Multiple  interconnected  tank  trainers  ma¬ 
neuvered  on  the  same  imaginary  battlefield  and 
cooperated  to  engage  a  common  enemy.  Their 
crews  sat  inside  tanklike  boxes  (see  figure  1)  and 
viewed  the  imaginary  battlefield  through  televi¬ 


sion  screens  mounted  where  the  vision  ports 
would  normally  be. 

DARPA’s  SIMNET  program  ended  in  1989, 
but  the  simulators,  communications  protocols, 
and  network  developed  by  the  program  are  still 
being  used  and  upgraded  by  ARPA’s  Advanced 
Distributed  Simulation  (ADS)  program,  the 
Army’s  Battlefield  Distributed  Simulation-De¬ 
velopmental  (BDS-D)  program,  and  other  De¬ 
partment  of  Defense  (DoD)  programs  (see  box  1). 
The  name  SIMNET  still  occurs  in  the  “SMNET/ 
Close  Combat  Tactical  Trainer”  line  item  in  the 
DoD  Comptroller’s  1995  Procurement  Programs 
(P-1)  report  to  Congress  (189).  The  term  is  still 
applied  to  simulators,  networks,  and  simulations 
that  still  use  the  SIMNET  communications  proto¬ 
cols,  which  have  been  superseded  by  the  Distrib¬ 
uted  Interactive  Simulation  (DIS)  protocols. 

A  distributed  interactive  simulation  is  “distrib¬ 
uted”  in  the  sense  that  simulator  computers  at 


'Hie  Advaflced  Research  Projects  Agency  was  estabUshed  in  1958,  renamed  the  Defense  Advanced  Research  Projects  Agency  in  1972, 
and  renamed  the  Advanced  Research  Projects  Agency  in  1993. 
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Driver’s  comparimerjt  of  a  SIMNET  Ml  tank  simulator.  The 
driver  stem  the  tank  with  handlebars  and  throttles  it  by 
rotating  one  of  the  handgrips,  much  tike  driving  a 
motorcycle,  while  viewing  the  virtual  world  outside  through 
three  vision  blocks-displays  of  computer-generated 
imagery  where  periscopes  would  be  on  a  real  ML  When  he 
drives  overstimulated  rough  terrain,  electromechanical 
actuators  move  his  seat  so  that  he  feels  bumps.  The  driver 
communicates  with  other  crew  members  over  an  intercom 
system  and  hears  simulated  engine  noise,  firing  of  the  tank's 
gun,  and  other  sounds  of  combat. 

multiple  training  sites  nationwide  are  connected 
by  means  of  a  local-area  network  (LAN;  see  fig¬ 
ure  2),  which  in  turn  maybe  connected  to  a  wide- 
area  network  (WAN)  such  as  the  the  Defense 
Simulation  Internet  (DSI)  (see  figure  3).  Trainees 
can  enter  disparate  tank,  helicopter,  or  other  simu¬ 
lators  and  all  participate  in  the  same  simulated 
combat.  The  simulation  is  “interactive”  in  the 
sense  that  humans  interact  with  the  computers  to 
influence  the  simulation.  The  battle  does  not  fol¬ 
low  a  set  script,  and  trainees  win  or  lose  on  the  ba¬ 
sis  of  their  performance.  The  trainees  fight  a 
similarly  networked  opposing  force,  which  may 
be  a  combination  of  virtual  forces-real  people 
operating  simulators-and  semiautomated  forces 
(SAF)-vehicles  and  weapons  simulated  by  com¬ 
puters  with  humans  providing  operational  super- 


Crew  compartment  of  a  SIMNET  Ml  tank  simulator.  The 
loader's  position  is  closest  to  the  camera,  the  gunner's 
position  beyond  and  to  the  left,  the  commander  stool 
beyond  and  to  the  right  The  loader  has  one  vision  block; 
the  gunner  has  one  that  shows  finer  detail,  and  the 
commander  has  several  vision  blocks,  but  cannot  open  the 
hatch  for  an  outside  view. 

vision.  In  both  the  virtual  forces  and  the 
semiautomated  forces,  human  behavior  is  simu¬ 
lated  by  humans,  albeit  humans  not  affectedly  the 
many  stresses  of  real  combat, 

These  simulations  hold  promise  beyond  their 
capability  to  train  equipment  operators  to  work  as 
a  team.  Ever  larger  numbers  of  networked  tanks, 
airplanes,  and  other  platforms  allow  higher  eche¬ 
lon  commanders  to  plan  operations  and  conduct 
them  in  simulation  before  conducting  them  in 
combat.  Proposed  weapon  systems  can  also  be 
simulated  in  order  to  evaluate  and,  if  necessary. 
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BOX  1:  SIMNET'sGeneoloc 


KEY: 

A2ATD:  Anti-Armor  Advanced  Technology  Demonstration 
ADCATT:  Air  Defense  Combined  Arms  Tactical  Trainer 
ADS:  Advanced  Distributed  Simulation 
AVCATT:  Aviation  Combined  Arms  Tactical  Trainer 
ARPA:  Advanced  Research  Projects  Agency 
BDS-D:  Battlefield  Distributed  Simulation-Developmental 
CATT:  Combined  Arms  Tactical  Trainer 
CCTT:  Close  Combat  Tactical  Trainer 


DARPA:  Defense  Advanced  Research  Projects  Agency 
DIS:  Distributed  Interactive  Simulation  [Project] 
ENCATT:  Engineering  Combined  Arms  Tactical  Trainer 
FSCATT:  Combined  Arms  Tactical  Trainer 
STOW-E:  Synthetic  Theater  of  War— Europe 
TGT:  Tank  Gunnery  T rainer 
TTGT :  Tank  Team  Gunnery  T rainer 


redesign  them — before  the  first  unit  is  built.  This 
technique  is  known  as  virtual  prototyping.'^ 
Creating  a  shared  virtual  battlefield  requires 
more  than  computers  and  display  equipment.  It  re¬ 
quires  collection  of  detailed  weapon,  terrain,  and 
climatic  data.  Some  simulators  can  use  data  col¬ 
lected  and  archived  for  other  purposes,  including 


platform  navigation  and  weapon  guidance,  open¬ 
ing  up  many  possibilities  to  improve  all  applica¬ 
tions  of  modeling  and  simulation  and  to  tie  them 
more  directly  to  actual  combat. 

High-tech  simulation  has  critics  as  well  as 
boosters.  Some  critics  have  seized  on  the  possibil¬ 
ity  of  “negative  learning”:  trainees  who  have  re- 


’ARPA’s  Simulation-Based  Design  (SBD)  (  157,158,159,  160)  and  TransTecli  (  161  )  programs  and  the  Army’s  current  Louisiana  Maneuvers 
(LAM)  program  (138,  199)  are  using  virtual  prototyping  of  proposed  military  equipment  to  reduce  acquisition  cost  and  time  and  to  increase 
operability  and  supportability. 

^There  are  many  nondefense  applications  of  virtual  prototyping.  Well-known  examples  include  Boeing  Aircraft  Company’s  virtual  proto¬ 
typing  of  its  new  777  commercial  transport  aircraft,  and  NASA’s  use  of  Locklieed’s  Preview  (for  “Prevent  Real  Errors  by  Virtual  Investigation 
of  Engineering  Worlds”  (5))  software  to  visualize  the  first  Hubble  Space  Telescope  servicing  mission,  which  identified  a  need  to  redesign  the 
Corrective  Optics  Space  Telescope  Axial  Replacement  (Costar)  (35,5  1,86,  137). 
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lOXI;  SIMNET'sGene 


DARPA’s  SIMNET  project  began  in  1983  to  exploit  technologies  and  concepts  developed  in  DAR- 
PA*s  Tank  Gunnery  Trainer  project’The  Army  became  a  cosponsor  in  1985  and  began  using  the 
technology  for  training  (SIMNET-T)  and  research  and  development  (SIMNET-D).  Meanwhile,  DARPA 
founded  the  Advanced  Distributed  Simulation  program  to  continue  technology  development  on  its  own. 
In  1989  DARPA  stopped  funding  the  SIMNET-T  and  SIMNET-D  projects,  which  two  years  later  became 
the  Army’s  Combined  Arms  Tactical  Trainer  (CATT)  project  and  Battlefield  Distributed  Simulation-Devel¬ 
opmental  project,  respectively.  The  CATT  project  is  a  collection  of  several  projects  to  develop  interoper¬ 
able  distributed  simulation  equipment  for  several  combat  arms.  Its  Close  Combat  Tactical  Trainer  proj¬ 
ect,  for  training  armored  units,  most  closely  resembles  the  early  SIMNET  project;  other  CATT  projects 
are  for  training  fire  support,  aviation,  air  defense,  and  engineering  units.  DARPA— now  called  ARPA— 
jointly  fund  the  Anti-Armor  Advanced  Tech,  .ology  Demonstration  program.  ARPA’s  Advanced  Distrib¬ 
uted  Simulation  program  manages  and  partially  funds  the  Synthetic  Theater  of  War  series  of  technology 
demonstrations.  The  Army’s  Distributed  Interactive  Simulation  project  coordinates  the  development  dis¬ 
tributed  interactive  simulation  protocols,  standards,  and  technologies  for  DoD-wlde  use. 

A  1992  study  by  the  Army  Science  Board  expressed  concern  that  the  Army’s  follow-on  programs 
to  SIMNET  were  fragmented— in  particular,  that  “CATT  and  BDS-D  are  being  pursued  as  separate,  in¬ 
dependent  efforts  rather  than  as  a  single,  integrated  program. 


'E.A.  Ailuisi,  "The  Development  of  Technology  for  Collective  Training:  SIMNET,  a  Case  History”  Human  Factors, 
33(3)343-362,  1991 

*U.S.  Army,  Army  Science  Board,  1991  Summer  Study  Final  Report,  Army  Simulation  Strategy  (Washington,  DC,  December 
1991),  p.  2. 

SOURCE:  Office  of  Technology  Assessment,  1995, 


relearned  to  rely  on  artifacts  of  the  simulations  may 
err  in  real  combat.  Other  critics  have  questioned 
the  process  by  which  the  high-tech  simulations 
have  been  deemed  valid  models  of  combat,  saying 
that  the  formalized  verification,  validation,  and 
accreditation  (VV&A)  process  needed  to  certify 
mathematical  models  of  combat  has  been  skipped 
in  the  case  of  the  high-tech  simulations.  DoD  is 
just  beginning  to  use  DIS  for  cost  and  operational 
effectiveness  analyses  (COEAs)  to  justify  budget 
requests  to  Congress  (and  within  DoD)  for  system 
acquisition  (156,138,199).  When  DIS  is  used  for 
such  predictive  uses,  validation  should  describe 
and  quantify  the  uncertainties  in  the  predictions.'* 


RECENT  DEVELOPMENTS 

The  following  DIS-related  developments  have  oc¬ 
curred  since  the  Senate  Committee  on  Armed  Ser¬ 
vices  held  hearings  on  simulation  in  1992  (145). 
They  are  discussed  further  in  either  this  back¬ 
ground  paper  or  a  previous  one  (147).  For  brevity, 
this  list  focuses  on,  but  is  not  limited  to,  official 
activities;  there  have  also  been  related  develop¬ 
ments  in  industry  and  academia. 

1992  Live  exercises  were  conducted  at  the  Wurt- 
smith  Air  Force  Base  Weapon  Storage  Area 
to  validate  the  Air  Force/Defense  Nuclear 


Tor  lists  of  questions  or  factors  to  consider  in  assessing  validity,  see  reference  number,  p.  281,  table  I;  reference  number(135),  p.  129; 
and  reference  number  (197),  p.  19,  table  2.1. 
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KEY:  CIG  =  Computer  Image  generator;  LAN  =  Local-area  network;  WAN  =  wide-ares  network. 
SOURCE:  Office  of  Technology  Assessment,  1995. 


Defense  Simulation  internet  experimental  integration  area  at 
the  Network  Engineering  and  Assessment  Facility  Laboratory 
of  the  Defense  information  Systems  Agency’s  Center  for 
Systems  Engineering  in  fteston,  Virginia. 


Agency  Security  Exercise  Evaluation  Sys¬ 
tem  (SEES)  (41,141);’ 

1993  Army  Pamphlet  5-11,  Verification,  Valida¬ 
tion,  and  Accreditation  of  Army  Models  and 
Simulations,  was  published; 

1994  DoD  Directive  5000.59,  “DoD  Modeling 
and  Simulation  (M&S)  Management,”  was 
issued; 

The  first  Modeling  and  Simulation  Technol¬ 
ogy  Area  Plan  (TAP)  was  prepared,  the  first 
Technology  Area  Review  (TAR)  was  con¬ 
ducted,  and  the  first  Technology  Area  As¬ 
sessment  (TAA)  was  prepared; 


’sees  version  2.0  Ls  a  multi-n.ser,  interactive, entity-level  constructive  simulation,  not  a  distributed  interactive  virtual  simulation.  However, 
the  method  used  to  validate  SEES  version  2.0  could  be  used  with  distributed  interactive  virtual  simulations. 


TO  VPC 


SOURCE:  Department  of  Defense,  Defense  Simulation  Internet  Joint  Project  Office.  1 994 
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The  first  of  six  planned  Anti-Armor 
Advanced  Technology  Demonstration 
(A2ATD)  experiments  was  conducted; 

The  SIMVAL’94  Simulation  Validation 
symposium  was  conducted  by  the  Military 
Operations  Research  Society  (MORS)  to  ad¬ 
dress  the  question,  “How  Much  Verification 
and  Validation  Are  Enough?” 

The  Synthetic  Theater  of  War-Europe 
(STOW-E)  exercise  was  conducted; 

1995  The  final  draft  of  the  DoD  Modeling  and 
Simulation  Master  Plan  (MSMP)  was-com- 
pleted  (195); 

A  handbook,  aimed  at  DIS  exercise  manag¬ 
ers,  on  methodology  for  VV&A  of  DIS,  is 
drafted  (175); 

Proposed  revisions  of  the  DIS  protocol  are 
discussed  at  the  12th  DIS  Standards  Work¬ 
shop;  and 

An  Architecture  Management  Group 
(AMG)  is  formed  to  steer  development  of 
&e  High-Level  Architecture  (HLA)  called 
for  by  the  draft  MSMP. 

SIMULATORS 

A  simulator  is  an  instrumented  mock-up,  such  as 
the  famous  Link  Trainer  for  pilots.  The  trainee  op¬ 
erates  the  simulator  in  much  the  same  way  that  he 
or  she  would  operate  the  vehicle  (or  other  equip¬ 
ment)  that  it  simulates.  An  instructor  may  pro¬ 
gram  the  simulator  to  present  the  trainee  with 
particular  tasks  or  problems  as  the  session  is  un¬ 
derway.  Simulators  are  useful  because  they  are 


The  first  Link  Trainer,  built  in  1929,  used  compressed  air  to 
make  the  cockpit  move  in  response  to  manipulation  of  the 
controls. 


cheaper  and  safer  to  operate  than  the  equipment 
that  they  simulate. 

The  variety  and  sophistication  of  simulators 
have  increased  since  the  Link  Trainer  was 
introduced  in  1929,  as  have  the  realism  and  costs 
of  top-of-the-line  simulators.  The  best  modem 
simulators  use  immense  computing  capacity  and 
exotic  display  equipment  to  synthesize  or  recon¬ 
struct  digitally  stored  scenes,  sounds,  and  vibra¬ 
tions,  giving  trainees  an  experience  so  vivid  and 
realistic  that  some  call  it  “virtual  reality” 
(182,147).  Still,  simulators  cost  less  than  would 
the  use  of  real  platforms  and  weapons  for  training. 
They  are  also  safer  and  provide  more  frequent  and 
varied  training  opportunities  than  actual  combat 
does. 

Less  expensive  models  lack  a  motion  base  (a 
cab  or  coc^^it  platform  moved  by  pistons  or  elec¬ 
tromechanical  actuators  to  provide  a  kinesthetic 
sense  of  being  there),  but  are  nevertheless  ade- 
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quate  for  most  training  goals  For  example,  the 
Air  Force,  following  the  lead  of  the  Air  National 
Guard,  is  buying  current  block  F-16C  simulators 
for  $650,000  each  for  use  as  unit  (or  unit-level) 
training  tevices  (UTDs).^  They  have  only  a  60-de¬ 
gree  vertical  by  78-degree  horizontal  field  of  view 
and  no  motion  base,  but  they  cost  only  $10,000 
each  per  year  to  maintain — a  thirtieth  the  cost  of 
the  most  recent  simulator — and  the  Air  Force  has 
concluded  that  they  can  meet  all  critical  training 
tasks  except  1)  post-merge  “fur  balls’’^  in  air-to- 
air  engagements,  2)  multiship  tasks,  and  3)  ma¬ 
neuvers  such  as  “the  pop”^^  in  air-to-ground 
missions  (168). 

It  appears  that  DoD  purchases  a  very  small 
fraction  of  the  output  of  the  domestic  simulator  in¬ 
dustry.  It  is  likely  that  most  of  DoD’s  purchases 


for  combat  modeling  and  simulation  are  not 
counted  in  simulator  sales. 

TYPES  OF  SIMULATIONS 

A  simulation  is  a  dynamic  information  constnact 
reflecting  some  larger  reality  or  possible  reality. 
Its  parts  and  processes  correspond  to  parts  and 
processes  of  the  lai'ger  reality,  albeit  with  less  than 
perfect  fidelity.  In  some  cases  the  correspondence 
involves  considerable  abstraction.  DoD  defines  a 
model  as  “a  physical,  mathematical,  or  otherwise 
logical  representation  of  a  system,  entity,  phe¬ 
nomenon,  or  process,”  and  a  simulation  as  “a 
method  for  implementing  a  model  over  time” 
(153).^^  DoD  also  distinguishes  three  major  cate- 


^  A  meta-analysis  of  26  experiments  on  flight  simulator  training  effectiveness  concluded  in  1992  tliat  “for  jet  training,  motion  cuing  was 
found  to  add  nothing  to  tlie  simulator  training  effectiveness/’  but  cautioned  that  “the  positive  effects  of  motion  for  any  one  task  may  have  been 
masked  by  the  negative  effects  of  motion  for  another  task”  (60),  Similarly,  a  1993  study  by  tire  U.S.  Army  Reseai  ch  Institute  concluded  tliat  no 
data  on  training  transfer  (i.e.,  effectiveness)  supported  theuse  of  motion-based  simulators,  possibly  because  motion  may  not  help  in  some  cases, 
or  possibly  because  of  insufficient  experimentation  or  shortcomings  in  experimental  technique  (8). 

The  Federal  Aviation  Administration  (FAA)  requires  a  flight  simulator  to  have  a  motion  base  capable  of  at  least  tliree  degrees  of  freedom 
(e.g.,  pitch,  roll,  and  yaw).  The  FAA  classifies  an  otherwise  similar  system  lacking  a  motion  base  and/or  a  visual  system  (e.g.,  a  computer  image 
generator  (CIG))  as  taflight  training  dmce.  An  FAA  Level  7  flight  training  device,  which  must  faitlifully  represent  tlie  instruments,  controls,  and 
responses  of  a  particular  aircraft,  can  be  used  for  all  training  for  which  an  FAA  Level  C  flight  simulator  can  be  used,  less  visual  and  motion 
aspects.  An  FAA  Level  C  flight  simulator  must  have  amotion  base  capable  of  six  degrees  of  freedom  (pitch,  roll,  yaw,  heave,  surge,  and  sway).  A 
Level  D  flight  simulator  must  have,  in  addition,  sound  simulation,  daylight  scene  generation  witli  at  least  400  edges  or  1,000  polygons,  and 
nighttime  scene  generation  with  at  least  4,000  points  of  light.  Experienced  crews  can  receive  training  credits  in  an  FAA  Level  D  flight  simulator 
to  become  type-rated  (i.e.  qualified)  to  tty  a  new  but  similar  type  of  aircraft  with  zero  flight  time.  The  U.K.  Civil  Aviation  Autiiority  has  similar 
regulations  (136). 

»  F-15C  UTDs  cost  $750,000  each. 

9  “Fur  ball”  is  fighter-pilot  jargon  for  aphase  of  an  engagement  of  several  friendly  aircraft  witli  several  opposing  aircraft  in  which  tlie  op¬ 
posing  groups  of  aircraft  are  merged  (mixed  together)  and  sorting  (the  assignment  of  targets)  is  especially  difficult. 

^  The  pop,  or  pop-up  is  a  maneuver  in  which  a  strike  aircraft  that  has  been  approaching  its  target  at  low  altitude  in  order  to  avoid  detection 
suddenly  climbs  in  order  to  acquire  (see  and  identify)  and  engage  the  target, 

^  ^  The  U.S.  Bureau  of  the  Census  reported  that  1993  sales  of  electronic  teaching  machines,  teaching  aids,  and  trainers  and  simulators  (prod¬ 
uct  code  36991)  totaled  $1.1  billion  ( 149).  Most  of  these  sales — about  $  1  billion — were  sales  of  electronic  trainers  and  simulators  (product  code 
36991  81).  Electronic  teaching  machines,  teaching  aids,  trainers  and  simulators  are  one  of  four  products  produced  by  the  “electrical  equipment 
and  supplies,  not  elsewhere  classified”  industry,  to  which  the  Office  of  Management  andBudgethas  as  signed  Standard  Industrial  Classification 
code  3699,  which  corresponds  to  the  1987  Input-Output  Account  580500  defined  by  the  U.S.  Bureau  of  Economic  Analysis  (BEA).  OTA’s 
round-by-round  analysis  (148)  of  the  BEA s  unpublished  six-digit  annual  Input-Output  tables  for  1987  (the  last  year  for  which  detailed  BEA  I-O 
tables  were  available  to  OTA)  indicated  that  the  federal  govermuent  puchased,  for  national  defense,  less  than  1  percent  of  tlie  industry’s  output 
(about  $3.5  billion;  cf.  (45))  directly  and  less  than  2  percent  indirectly  (double- counting  value  added  in  different  tiers). 

^^Paul  Davis,  Corporate  Research  Manager  at  the  RAND  Corporation,  considers  tlie  terms  model  and  simulation  to  be  “essentially  synony¬ 
mous,”  both  meaning  “almost  any  kind  of  representation  of  reality.”  He  notes  that  “few  terms  are  used  consistently  across  communities  or 
even  from  day  to  day  within  a  given  community”  (24). 


Distributed  Interactive  Simulation  of  Combat  1 11 


The  Army’s  Louisiana  Maneuvers  of  the  early 
1940s  (44),  the  U.S.  Navy’s  mock  attack  on  Pearl 
Harbor  just  before  the  real  one  (100),  and  portions 
of  recent  REFORGER,  Team  Spirit,  and  Atlan¬ 
tic  Resolve  exercises  are  all  examples  of  live  sim¬ 
ulation.  Most  field  exercises  are  scripted  in 
advance. 

Though  usually  a  means  of  training,  exercises 
can  be  used  for  research.  For  example,  the  Louisi¬ 
ana  Maneuvers  that  preceeded  U.S.  involvement 
in  World  War  II  taught  the  U.S.  Army  important 
lessons  about  how  to  handle  large  armored  forma¬ 
tions.^^ 

The  Army’s  National  Training  Center  (NTC) 
offers  the  most  realistic  land-warfare  exercise  en¬ 
vironment  available  today.  Instrumented  Ameri¬ 
can  fighting  men  and  machines,  including  some 
playing  the  role  of  opposing  forces  (OPFOR),  do 
battle  on  a  desert  battlefield  the  size  of  Rhode  Is¬ 
land  using  weapons  that  fire  coded  laser  pulses 
instead  of  bullets.  The  pulses  specify  the  type  of 
shot  simulated,  so  that  a  tank  receiving  a  rifle  shot 
knows  that  it  is  unhurt,  while  a  tank  hit  with  a 
TOW*^  antitank  guided  missile  knows  that  the 
time  has  come  to  simulate  its  own  demise  by  shut¬ 
ting  off  its  systems  and  emitting  smoke.  Training 
at  NTC  was  given  major  credit  for  the  success  of 
U.S.  forces  in  the  Persian  Gulf  War. 

Live  simulation  is  used  by  all  the  services,  as 
well  as  by  the  U.S.  Unified  Combatant  Com¬ 
mands  and  other  organizations.  Air  warfare,  naval 
warfare,  amphibious  warfare,  and  joint  (i.e.,  mul¬ 


tiservice)  and  theater  warfare  are  simulated.  The 
U.S.  Navy’s  air  combat  simulation  conducted  at 
Naval  Air  Station  Monterrey,  Cahfornia,  was  pub¬ 
licized  by  the  motion  picture  Top  Gun.  Less  well 
known,  perhaps,  are  the  Red  Flag  and  Green  Flag 
exercises  conducted  at  the  Air  Force  Fighter 
Weapons  School  at  Nellis  Air  Force  Base,  Neva¬ 
da,  which  simulate  air-to-air,  strike,  and  surface- 
to-air  combat,  as  well  as  electronic  warfare.  There 
are  many  other  periodic  and  ad  hoc  live  simula¬ 
tions. 

Military  exercises,  especially  those  conducted 
abroad  or  on  the  high  seas,  serve  diplomatic  and 
other  purposes  beyond  their  utility  to  the  military. 
REFORGER  and  Team  Spirit  are  prime  examples 
in  this  regard:  these  periodic  exercises  help  main¬ 
tain  close  U.S.  ties  with  the  Federal  Republic  of 
Germany  and  the  Republic  of  Korea,  respectively. 

I  Constructive  Simulation 

Prominent  historical  examples  of  constructive 
simulations  include  exercises  on  maps  and  sand 
tables,*^48  games  with  model  ships  played  at 
the  U.S.  Naval  War  College^^.  Computer-assisted 
exercises  (CAXs)  have  largely  superseded  sand- 
table  and  map  exercises.  In  some  cases  they  pit  a 
commander  (often  with  staff  and  subordinate 
headquarters)  against  a  similarly-equipped  hu¬ 
man  playing  the  role  of  an  opposing  commander. 
In  other  cases,  there  are  no  humans  playing  mili¬ 
tary  roles.  Instead,  an  analyst  enters  data  about  a 


REFORGER  stands  for  Return  of  Forces  to  Gennany.  REFORGER  exercises  have  been  superseded,  in  die  post- Cold  War  era,  by  Atlantic 
Resolve  exercises. 

Some  years  earlier  German  forces  conducted  major  exercises  in  the  course  of  developing  die  Blitzkrieg  approach  to  warfare.  Interesting¬ 
ly,  these  exercises  began  before  tanks  were  available:  crews  operated  cars  fitted  with  tanklike  bodies  (17), 

A  tube-launched,  optically  tracked,  wire-command  link  guided  missile. 

^  These  older  methods  are  still  used.  For  example,  to  visualize  and  rehearse  their  part  in  Operation  Desert  Storm,  one  U.S .  Anny  tank  battal¬ 
ion  commander  and  his  staff  maneuvered  military  models,  which  diey  had  purchased  in  garrison  in  Germany,  on  Saudi  Arabian  sand — sans 
table  (206).  The  Army  history  of  the  Persian  Gulf  War  shows  another  example  (123), 

^  ^  The  Army  may  use  “virtual  sandtables”  in  the  future.  The  Army’s  Communications  and  Electronics  Command  (CECOM)  is  funding  die 
development  of  an  architecture  for  a  “VR-based  dispersed  command  post. . . ,  where  each  commander’s  virtual  environment  integrates  a  virtual 
conference  room  containing  virtual  video  walls,  battlemap  projections,  interactive  3D  sandtable  models,  and  visual  representations  ot  remote 
conference  participants”  (47), 

^^See,  for  example,  (105),  especially  pp.  73ff;  (100);  and  (110)  . 
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This  map  display  shows  part  of  a  tank  battle  being  simulated  by  an  early  version  of  the  Modular  Semi— Automated  Forces  (ModSAF)  software  A 
constructive  simulation,  it  simulates  movement  and  combat  of  individual  entities  (vehicles  and  soldiers)  and  displays  the  location  and  status  of 
individual  vehicles,  as  shown  here,  or  small  units,  such  as  tank  platoons  and  companies,  An  operator  may  be  privileged  to  give  orders  (e  g  ,  move  to 
contact,  or  move  and  wait)  to  various  units.  Use  of  ModSAF  is  a  computer-assisted  exercise  and  maybe  a  command  post  exercise  (CPX)  for  the 
benefit  of  unit  commanders  and  their  staffs.  A  computer  ainning  ModSAF  can  participate  in  a  distributed  interactive  simulation  by  sending  specially 
formatted  messages  reporting  the  location  and  status  of  each  entity  that  it  simulates,  and  by  processing  similar  messages  from  other  computers  (or 
manned  simulators  or  Instrumented  vehicles)  to  determine  whether  the  entitles  or  events  that  they  simulate  should  be  displayed  or  should  Influence 
the  entities  simulated  locally 
SOURCE:  The  RAND  Corp.,  1995, 


real  or  hypothetical  situation  into  a  computer  and 
then  instructs  the  computer  to  predict,  step  by 
step,  what  might  happen,  in  some  cases  with  reso¬ 
lution  (details)  of  individual  vehicles.  This,  too,  is 
a  constructive  simulation  (see  figure  4). 

Command  post  exercises  (CPXs)  and  their  na¬ 
val  equivalents.  Fleet  Exercises  (FLEETEXs)  ex¬ 


ercise  only  commanders,  their  staffs,  and  the 
communications  system.  The  forces  that  they  pre¬ 
tend  to  command  appear  only  as  marks  on  a  map 
or  chart  display.  Procedural  CPXs  are  often  heavi¬ 
ly  scripted  to  insure  that  certain  procedures  are  re¬ 
hearsed  or  tested;  scripts  may  branch  at  certain 
points  according  to  command  decisions  made  by 
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I  TAR!  F  t:  U.S.  Army  Command  Fr.helnpR  I 

Echelon 

Notation 

Commander’s  rank 

Army  Group 

xxxxx 

General 

Army 

xxxx 

General 

Corps 

XXX 

Lieutenant  General 

Division 

XX 

Major  General 

Brigade 

X 

t 

Brigadier  General 

Group/Regimenf 

m  ; 

Colonel 

Battalion' 

n 

Lieutenant  Colonel 

Company" 

Captain 

Platoon 

••• 

2d  or  1st  Lieutenant 

Squad 

•• 

Sergeant 

Section 

• 

Corporal 

Individual 

Private  or  Private  First  Class 

‘Typical. 

“Regiment,  if  a  cavalry  unit. 

'Squadron,  if  a  cavalry  unit. 

'‘Troop,  if  a  cavalry  unit;  battery,  if  an  artillery  unit. 

SOURCES:  J.F.  Dunnigan,  How  to  Make  War-A  Comprehensive  Guide  to  Modem  Warfare,  completely  revised  edition  (New  York,  NY:  William 
Morrow,  1988)  ;  U.S.  Army,  Operations,  FM  100-5,  1986;  U.S,  Department  of  Defense,  Common  Warfighting  Symbology  Version  1,  MIL- 
STD-2525,  Sept.  30, 1994 


the  participants.  CPXs  are  both  live  and  construc¬ 
tive  simulations.  The  participants  in  one  com¬ 
mand  post  (CP)  use  some  of  the  command  and 
control  equipment  that  they  would  use  in  war: 
field  telephones,  computer  terminals,  maps  or 
map  displays,  and  sometimes  radios.  In  this  re¬ 
spect  a  CPX  is  a  live  simulation.  The  equipment 
connects  the  CP  not  only  to  others  in  the  CPX  but 
also  to  computers  (if  a  CAX)  or  referrees  who  per¬ 
form  conshuctive  simulation  of  the  forces  that 
those  in  the  CP  command.  In  a  different  inter¬ 
pretation,  the  command  and  control  system  is  con¬ 
sidered  to  be  a  simulator,  not  just  a  means  of 
communicating  with  one.  In  this  sense  a  CPX  is 
a  virtual  simulation.  Because  it  is  problematic,  if 
not  impossible,  to  classify  a  CPX  (and  other  simu¬ 
lations)  as  live,  constructive,  or  virtual,  Paul  Da¬ 
vis  has  proposed  a  generalized  classification 
scheme,  which  includes  the  hybrid  classes 
constructive/virtual  (CV),  virtual/live  (VL), 
constructive/live  (CL),  and  constructive/virtual/ 
live  (CVL)  (24). 


Simulations  of  all  these  types  differ  from  one 
another  in  scope,  focus,  abstraction,  and  aggrega¬ 
tion.  One  training  task  will  require  a  particular  ac¬ 
tivity  (its  focus)  to  be  simulated  in  detail  but  can 
tolerate  greater  abstraction  in  the  simulation  of  a 
related  activity,  which  might  be  the  focus  of  a  dif¬ 
ferent  simulation.  Concreteness  and  detail  are 
necessary  to  make  a  tank  simulator  seem  realistic 
to  the  crew  using  it,  but  the  simulator  need  not 
show  forces  more  than  20  km  away  or  correctly 
predict  which  side  will  win  if  the  only  objective  of 
the  simulation  is  to  improve  the  combat  skills  of 
participating  tank  crews.  In  contrast,  both  in  exer¬ 
cises  and  war,  commanders  and  their  staffs  (see 
table  1)  deal,  with  aggregated  forces.  They  do  so 
through  abstraction:  an  armored  battalion,  con¬ 
sisting  of  about  a  thousand  armored  vehicles  and 
about  16,000  soldiers  at  full  strength  (see  table  2), 
is  often  represented  on  paper  maps  and  computer 
displays  at  higher  echelon  headquarters  as  a  rect¬ 
angle  (any  unit)  enclosing  an  oval  (representing 
tank  tracks,  hence  armor)  with  two  “x”s  on  top 


14  Distributed  Interactive  Simulation  of  Combat 


TABLE  2:  Typical  Equipment  ol  U.S.  Army  Armored 
Divisions  and  Mechanized  Intantry  Divisions'’ 


Armored 

Division 

Mechanized 

infantry 

Division 

Transport 

3,500 

3,500 

Soldiers 

16,300 

16,600 

Tanks 

348 

290 

Armored  personnel 
carriers 

652 

727 

Artillery 

143 

143 

Anti-tank  guided 
munitions 

523 

660 

Machine  guns 

1,360 

1,616 

Heavy  mortars 

66 

66 

Anti-aircraft  guns 

24 

24 

Self-propelled  SAMs 

24 

24 

Light  SAMs 

60 

60 

Anti-tank  rocket 
launchers 

900 

1,000 

KEY:  SAM  =  surface-to-air  missile. 

‘This  is  a  simplified  table.  Terms  for  vehicles  and  weapons  are  not 
necessarily  official  U.S.  Army  terms.  Numbers  of  personnel,  ve¬ 
hicles,  and  weapons  approximate  those  of  units  at  full  strength. 
SOURCE:  J.F,  Dunnigan,  How  to  Make  War-A  Comprehensive 
Guide  to  Modem  Warfare,  completely  revised  edition  {New  York,  NY: 
William  Morrow,  1988) 

(representing  the  division  level  of  aggregation). 
An  armored  battalion,  consisting  of  tens  of  ve¬ 
hicles  and  hundreds  of  soldiers  (see  table  3), 
would  be  represented  by  the  same  symbol,  except 
with  two  vertical  bars  (instead  of  two  “x”s)  on  top 
of  the  rectangle.  A  mechanized  infantry  battalion 
with  about  700  soldiers  (see  table  4)  would  be  rep¬ 
resented  by  the  same  symbol,  with  an  “x”  (repre¬ 
senting  crossed  rifles)  criss-crossing  the  rectangle 
enclosing  the  oval.  Messages  are  equally  abstract: 
“US  lAB”  would  represent  the  U.S.  Army’s  First 
Armored  Battalion. 

Whatever  else  they  do,  combat  models  model 
aspects  of  individual  and  collective  human  behav¬ 
ior,  such  as  operational  decision  making.  This 
modeling  has  dways  been  controversial,  because 
human  behavior  cannot  be  reduced  to  a  set  of  prin¬ 
ciples  as  simple  and  reproducibly  verifiable  as 
those  of,  say,  aerodynamics  or  sohd  mechanics. 


TABLE  3:  U.S.  Tank  Unit  Organization 


Unit 

Tanks 

Soidiers  Subordinate  Units 

Battalion 

33-60 

182-540  3  tank  companies, 

1  headquarters, 
other  units. 

Company 

10-22 

40-88 

3-4  tank  platoons, 

1  headquarters 

Platoon 

3-5 

12-20 

3-5  tanks 

SOURCES:  J.F.  Dunnigan,  How  to  Make  War— A  Comprehensive 
Guide  to  Modem  Warfare,  completely  revised  edition  (New  York,  NY: 
William  Morrow,  1988);  U.S.  Army,  Operations,  FM  100-5, 1986 


TABLE  4:  U.S.  Mechanized  Infantry 

Unit  Organization 

Unit 

Soidiers  Subordinate  Units 

Battalion 

696 

6  companies:  4  infantry,  1 
antitank,  1  headquarters 

Company 

111 

3  infantry  platoons,  1 
headquarters  section 

Platoon 

33 

3  infantry  squads,  1 
headquarters 

Squad 

9 

9  individuals 

SOURCES:  J.F.  Dunnigan,  How  to  Make  War— A  Comprehensive 
Guide  to  Modem  Warfare,  completely  revised  edition  (New  York,  NY: 
William  Morrow,  1988);  U.S.  Army,  Operations,  FM  100-5, 1986. 


which  are  used  to  model  and  simulate  the  response 
of  an  airplane  to  a  pilot’s  control  actions.  Combat 
is  a  much  more  complex  and  less  predictable  pro¬ 
cess,  the  result  of  actions  by  many  humans.  Of 
necessity,  modeling  them  requires  many  simplify¬ 
ing  assumptions,  often  without  any  scientific  jus¬ 
tification. 

The  uncertainty  and  unpredictability  in  combat 
is  further  complicated  today  with  the  shift  from  a 
defined  superpower  confrontation  to  more  com¬ 
plicated  and  ambiguous  tasks  for  the  military. 
Stuart  Starr,  Director  of  Plans  at  the  MITRE  Cor¬ 
poration,  has  observed  that  because  of  geopoliti¬ 
cal  changes  in  the  last  5  years,  “models  of 
strategic  and  high  intensity  conventional  con¬ 
flicts”  are  of  less  interest.  In  their  place,  “the 
M&S  community  is  being  confronted  with  the 
need  to  develop  credible  models  of  potential 
crises,  humanitarian  assistance,  and  conflicts  with 
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This  compuier-generatad  scene  is  analogous  to  scenes  presented  to  pilots  ki  flight  simulatore.  The  scene  actually  illustrates(although  not  precisely) 
an  episode  of  live  simulation  in  the  November  1994  Synthetic  Theater  of  War  Europe  exercise,  in  which  U.S.  Air  Force  F-ISC  Eagles  and  US.  Navy 
F/A-18  Hornets  practiced  joining  procedures.  Their  positions  were  trackad  by  the  Tactical  Aircrew  Combat  Training  System  at  the  Marine  Corps  Air 
Station  at  Cherry  Point,  North  Carolina,  end  relayed  to  participating  vehicle  and  radar  simulators  and  constructive  simulations. 

SOURCE:  Office  of  Technology  Assessment,  1995. 


and  among  third  world  nations.  These  demands 
will  stress  the  technology  communities’  abilities 
to  model  human  behavior  and  to  represent  a  broad 
spectrum  of  environmental  conditions”  (131). 

I  Virtual  Simulation 

Some  DIS  proponents  argue  that  constructive 
simulations  often  do  not  simulate  human  behavior 
well,  and  that  the  solution  to  this  problem  is  to  put 
human  soldiers  (but  not  their  vehicles  and  weap- 
-ons)  back  into  simulation.  Partly  for  this  reason 
(validity)--but  mainly  because  it  was  designed  to 
train  small  units--SIMNET  put  human  soldiers, 
by  the  company,  into  a  computer  simulation  and 
tried  to  evoke  realistic  individual  and  organiza¬ 
tional  behavior  by  presenting  them  with  a  realis¬ 
tic,  threatening  tactical  environment,  using  sights 
and  sounds  to  raise  the  tension  level  as  much  as 
conveniently  possible  (see  box  2).  The  sight-and- 
sound  presentation  created  by  SIMNET  is 
counted  by  many  as  “virtual  reality”  or  a  “syn¬ 
thetic  environment”  (see  figure  5)  (182,147). 

Participants  in  a  DIS  exercise  can  get  very  im¬ 
mersed  in  the  experience.  During  an  early  SIM¬ 
NET  exercise,  a  tank  commander  dove  out  of  his 
tank  to  take  a  compass  reading,  as  he  had  been 
trained  and  had  grown  accustomed  to  get  out  of  a 


real  steel  tank  in  order  to  get  an  accurate  magnetic 
compass  reading,  unaffected  by  the  magnetic  ef¬ 
fect  (permeability)  of  the  tank.  When  he  realized 
where  he  was  and  what  he  had  done,  he  went  back 
into  the  simulator  (95). 

THE  EVOLUTION  OF  DISTRIBUTED 
INTERACTIVE  Simulation 

Computer-mediated  distributed  interactive  simu¬ 
lation  can  be  traced  to  the  Air  Force’s  Semi-Auto¬ 
matic  [sic]  Ground  Environment  air  defense 
system,  which  was  developed  during  the  1950s 
(147).  Virtual  simulation  can  be  traced  to  the  first 
Link  Trainer,  demonstrated  in  1929  (147),  al¬ 
though  rehearsing  or  playing  at  war  with  props  for 
weapons-a  form  of  virtual  simulation-is 
doubtless  prehistoric.  However,  modem  distrib¬ 
uted  interactive  simulation  began  with  DARPA’s 
SIMNET  program 

SIMNET 

SIMNET,  a  billion-dollar  (194)  development  pro¬ 
gram  of  DARPA  (now  ARP  A)  and  the  Army,  took 
existing  simulation  technology  and  applied  mi¬ 
croprocessing  and  networking  technology  to 
achieve  an  interactive  simulation.  SIMNET  simu¬ 
lates  the  combined  aims  battlefield,  allowing 
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BOX  2;  Human  Factors  Engineering 


SIMNET  marked  a  shift  in  simulator  design  from  an  engineering  or  physical  reproduction  ap¬ 
proach  in  the  tradition  of  the  Link  Trainers  and  flight  simulators  to  manipulating  “brain  state”  and  relying 
more  on  human  factors  analysis.  Bringing  the  psychological  realm  of  human  behavior  Into  simulation 
enabled  designers  to  understand  just  which  sensory  cues  to  give  the  trainee  in  order  to  move  his  men¬ 
tal  focus  past  the  physical  mechanism  he  occupied  to  the  virtual  battlefield.  “The  design  did  not  con¬ 
centrate  on  the  armored  vehicle  per  se;  rather...  design  became  behaviorally  driven  and  began  by  iden¬ 
tifying  cues  that  crew  members  needed  to  learn  their  specific  duties  and  tasks. 

The  entertainment  industry  has  long  been  expert  at  using  just  those  sounds  and  sights  that  en¬ 
gage  the  imagination  of  the  audience  while  eliminating  any  distraction  that  would  break  the  “suspension 
of  disbelief.  ’’"Some  of  these  techniques  were  used  in  the  SIMNET  simulators.  For  example,  the  sound 
of  the  tank  firing  was  enhanced  with  overtones  from  a  tiger’s  growPto  add  an  adrenaline  rush  to  en¬ 
hance  the  realism  of  the  simulation. 

The  difference  between  the  Link-style  approach  and  the  SIMNET -style  approach  is  reflected  in 
two  different  concepts  of  fidelity.  Fidelity  in  the  simulation  world  means  much  the  same  thing  as  it  does 
in  the  audio  world,  A  “high  fidelity”  stereo  system  duplicates  the  real  sound  so  that  it  is  hard  to  tell  the 
difference.  The  effectiveness  of  simulators  has  traditionally  been  measured  the  same  way — by  how 
close  the  experience  felt  to  the  real  thing.  In  SIMNET,  however,  It  was  found  that  fidelity  or  “accuracy  in 
details”'was  not  necessarily  important  to  achieve  immersion  that  was  necessary  for  teaching  skills  and 
sometimes  not  as  effective  as  specific  cueing,  a  concept  the  SIMNET  team  called  selective  fidelity. 

Instead  of  starting  with  a  complete  cockpit  and  subtracting  the  extraneous  components,  the  SIM¬ 
NET  team  started  with  nothing  and  built  a  simulated  tank  interior  based  on  what  they  felt  they  needed  to 
cue  a  desired  behavior.  The  beginning  of  the  design  process  was  a  task  list  of  what  skills  they  wanted 
to  teach.  Nothing  was  added  to  the  design  that  did  not  contribute  to  eliciting  some  specific,  desired 
training  behavior.'This  approach  resulted  in  a  lower  fidelity  and  much  cheaper  simulator,  which  made  it 
possible  to  produce  large  number  of  simulators  to  network  together. 


'E.A.  Alluisi,  ‘The  Development  of  Technology  for  Collective  Training:  SIMNET,  A  Case  History,”  Human  Factors,  33(3): 
343-362,  1991. 

*J.  Isdale,  “What  is  Virtual  Reality?  A  Homebrew  Introduction,"  March  1993,  available  Online  at  <URL:ftp://sunsite.unc.edu/ 

pub/academic/computer-science/virtual-reality/papers/whatisvr.txt>. 

'’C.  Portman,  Hollywood  soundman,  interview  with  L.  Voss  at  Perceptronics,  Inc.,  Woodland  Hills,  CA,  August  1992, 

*  Webster's  New  Collegiate  Dictionary  (Springfield,  MA:  G.  &  C.  Merriam  Company,  1974). 

*J.  McDonough,  interview  with  L.  Voss  at  Illusion  Engineering  Inc.,  Westlake  Village,  CA,  August  1992. 


forces  as  large  as  battalions  to  fight  opposing 
forces  of  similar  size;  it  provides  an  alternative  to 
combined  arms  field  exercises.  The  network  can 
link  hundreds  of  simulators  at  widely  dispersed 
sites  in  real  time.  By  the  end  of  the  project  almost 
300  simulators — Ml  Abrams  tanks,  M2  and  M3 
Bradley  fighting  vehicles,  aircraft,  and  command 
posts  (2) — ^had  been  produced. 

The  hardware  for  a  SIMNET-type  simulator  in¬ 
cludes  a  computer  (called  a  simulation  host),  an 
image  generator,  a  sound  system,  and  network 


connections,  as  shown  in  figure  2.  The  image  gen¬ 
erator  is  a  special-purpose  computer  that  pro¬ 
cesses  the  simulation  data  and  generates  computer 
images  of  the  action.  The  simulation  hosts  are 
connected  via  local-  and  wide-area  networks  to 
other  simulators  engaged  in  the  same  battle.  When 
the  tank  simulator  is  networked  into  a  battlefield 
simulation,  the  crew  see  the  action  of  the  battle¬ 
field  in  computer-generated  scenes  through  win¬ 
dows  or  portals  at  their  stations.  The  actions  by 
their  tank  simulator — fire  and  maneuver — are  re- 
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BOX  2:  Human  Factors  Engineering  (Cont'd.) 


What  the  SIMNET  designers  discovered  was  that  faithful  reproduction  of  the  physical  details  of 
the  equipment  and  even  the  battlefield  visuals  were  less  important  than  the  action  itself  in  engaging  the 
battlefield  behavior.  Encountering  other  combatants  in  a  battlefield  environment  elicited  a  combative 
response. 

A  psychologically  different  interaction  results  when  trainees  become  aware  that  they  are  working 
with  others  on  a  battlefield,  whether  cooperatively  with  a  wing  man  or  in  opposition  with  an  enemy  pla¬ 
toon.  The  importance  is  in  projecting  one’s  conscious  thinking  process  onto  the  battlefield.® 

In  SIMNET  the  battlefield  radio  communication,  group  skills,  tactics,  and  plans  for  engaging  the 
enemy  are  implemented  in  real  time  in  an  environment  that  approximates  the  experience  of  the  battle¬ 
field  under  controlled  circumstances.  The  purpose  is  to  train  advanced  skills.  SIMNET  trains  soldiers 
who  are  already  proficient  in  individual  skills  (such  as  driving  and  gunnery)  the  next  level  of  skills,  which 
is  working  together  as  a  unit.  At  this  advanced  level,  the  fidelity  of  system  and  environmental  represen¬ 
tations  is  less  important  than  the  fidelity  of  the  human  interaction.  Interactive  simulation  enables  soldiers 
to  learn  group  fighting  skills  and  enables  commanders  to  Improve  tactical  and  strategic  decision  mak¬ 
ing. 

A  SIMNET  simulator  has  several  video  slots,  each  portraying  that  part  of  a  battlefield  a  war  fighter 
would  see  looking  through  a  corresponding  window  of  a  cockpit.  Early  in  the  development  of  SIMNET 
instructors  noted  that  shortly  after  the  beginning  of  a  simulation  session,  an  experienced  war  fighter 
ceases  to  notice  gaps  or  seams  in  the  displayed  scene,  because  he  easily  projects  out  into  the  battle¬ 
field  and  is  absorbed  in  the  experience  and  the  interaction  with  other  people.  An  inexperienced  war 
fighter,  on  the  other  hand,  does  not  recognize  the  pattern  of  battle  unfolding  as  easily  and  remains  dis¬ 
tracted  by  the  imperfections  of  the  simulation.' 

It  remains  a  tenet  of  successor  programs,  such  as  the  Army’s  Close  Combat  Tactical  Trainer  proj¬ 
ect,  that  the  important  aspect  of  battlefield  fidelity  is  the  percentage  of  desired  behavior  learned.  This 
tenet  shifts  the  focus  from  the  machinery  to  the  human  interaction. 


‘H.  Crooks,  interview  with  L.  Voss  at  Illusion  Engineering  Inc.,  Westlake  Village,  CA,  August  1992. 
nbid. 

SOURCE:  Office  of  Technology  Assessment,  1995. 


fleeted  in  the  other  simulators,  too,  and  replicated 
in  sound  and  sight.  If  one  tank  is  hit  by  another,  the 
damage  is  calculated  by  the  tank  that  was  hit  and 
reported  back  through  the  network.  The  image 
generator  generates  scenes  of  the  rough  geograph¬ 
ical  features  of  the  battlefield-hills,  blue  or 
cloudy  skies,  trees,  houses — and  the  features  of 
battle— enemy  tanks,  perhaps  hiding  behind  trees 
in  the  distance;  smoke  and  fire;  helicopters  over¬ 
head;  and  the  boom  of  mortar  fire. 

SIMNET  evolved  Irom  a  1979  DARPA  proj¬ 
ect,  the  tank  gunnery  trainer  (TGT).  The  TGT  was 
designed  to  use  an  Apple  II  computer  to  overlay 
computer-generated  imagery  (CGI)  on  back¬ 


ground  imagery  played  back  from  a  videodisc. 
This  innovative  approach,  which  was  demon¬ 
strated  in  late  1979,  promised  to  provide  gunnery 
practice  at  a  much  lower  cost  than  range  gunnery 
practice.  The  TGTs  were  expected  to  cost  less  than 
$10,000  in  quantity. 

In  1981,  Jack  Thorpe,  an  Air  Force  officer,  be¬ 
came  manager  of  the  TGT  program.  He  proposed 
networking  five  TGTs  to  provide  a  low-cost  pla¬ 
toon-level  training  device,  a  tank  team  gunnery 
trainer  (TTGT),  in  which  five  trainees  would  be 
shown  the  same  seene  and  would  compete  to  be 
the  first  to  sight  and  fire  at  an  enemy  tank.  He  also 
proposed  a  different  type  of  networking  that 
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would  perniit  large  numbers  of  simulators  to  be 
networked  so  that  they  could  interact — e.g.,  so 
that  the  crew  in  one  simulator  would  see  an  image 
of  tanks  operated  by  the  crews  of  other  simulators 
moving  on  the  displayed  virtual  battlefield.^^  In 
1982  DARPA  awarded  contracts  directed  toward 
this  goal  and  also  approved  the  SIMNET  project, 
an  extension  of  it  focused  on  Army  collective 
training  needs  (2). 

The  first  SIMNET  contracts  were  awarded  in 
1983.  At  that  time  the  network  architecture  con¬ 
cept  stiU  envisioned  generating  imagery  for  each 
simulator  centrally  and  transmitting  it  over  the 
network  to  the  simulator.  It  was  soon  discovered 
that  this  would  be  too  costly,  but  it  also  appeared, 
at  least  to  Thorpe,  that  microprocessors  capable  of 
rendering  a  simulator’s  imagery  would  soon  be 
available  and  affordable  enough  to  put  a  computer 
image  generator  (CIG)  in  each  simulator.  The 
links  of  the  planned  simulator  network  would  then 
not  need  the  high  information-carrying  capacity 
required  to  transmit  imagery;  instead,  each  simu¬ 
lator  would  only  need  to  broadcast  a  short  mes¬ 
sage  (a  packet^i  or  datagram^^)  on  the  network 
every  second  or  so  to  notify  other  simulators  of  its 
position  and  certain  other  attributes  Oheading,  ve¬ 
locity,  turret  azimuth,  gun  tube  elevation,  etc.). 
The  computer  in  each  other  simulator  would  ex¬ 
amine  each  datagram  to  decide  whether  the  data¬ 
gram  was  from  a  tank  within,  entering,  or  leaving 
its  simulated  field  of  view  (2).  If  so,  the  simula¬ 
tor’s  CIG  would  make  the  appropriate  change  in 
the  scene  that  it  displayed. 


A  breadboards^  version  of  the  SIMNET  CIG 
was  demonstrated  late  in  1984.  In  January  1985  a 
prototype  Ml  tank  simulator,  made  of  plywood, 
was  demonstrated  to  the  Secretary  of  the  Army, 
the  Chief  of  Staff,  Army,  and  other  Army  officials. 
Army  interest  and  funding  continued.  Later  that 
year  two  developmental  Ml  simulators  were  net¬ 
worked  together  at  the  annual  convention  of  the 
Association  of  the  U.S.  Army.  By  May  1986,  two 
production-model  Ml  simulators  were  net¬ 
worked,  and  by  October  1986,  eight  Ml  simula¬ 
tors — ^two  platoons— at  Fort  Knox,  Kentucky, 
had  been  networked  with  computers  that  repre¬ 
sented  artilleiy  fire  support  elements,  close  air 
support  aircraft,  a  tactical  operations  center,  and 
an  administrative  and  logistical  operations  center. 
In  November  1987,  two  attack  aircraft  simulators 
were  added  to  the  net.  On  simulator  displays, 
these  looked  like  hybrids  of  A- 10  and  F-16  air¬ 
craft,  so  exercise  participants  and  observers  jok¬ 
ingly  dubbed  the  simulated  aircraft  “A-13s,” 
because  13  is  the  average  of  10  and  16. 

By  the  end  of  1987,  the  first  battalion-level 
training  exercises  had  been  conducted  with  five 
air  defense  simulators  and  two  helicopter  simula¬ 
tors  added,  bringing  the  total  number  of  simula¬ 
tors  at  Fort  Knox  up  to  56  fully  interactive  ground 
vehicle  and  aircraft  simulators  (2).  By  February 
1988,  the  SIMNET  facility  contained  a  total  of  71 
interactive  simulators.  Battalion-level  force-on- 
force  operational  exercises  with  Forward  Area  Air 
Defense  elements  were  conducted  in  April.  A  test 


^**10 1978,  while  working  at  the  Air  Force  Office  of  Scientific  Research,  Thorpe  had  proposed  a  similar  networking  of  flight  simulators  (2). 

A  packet  is  aphysical  (e.g.,  electrical  or  optical)  signai  composed  of  a  sequence  of  pulses,  representing  1  s  and  Os,  in  a  certain  order:  first  a 
Special  sequence  of  pulses  {aflag)  marking  the  beginning  of  the  packet;  then  a  header  describing  the  packet  type  or  format,  the  address  of  the 
computer  or  computers  for  which  it  is  intended,  and  in  some  cases  a  number  used  by  the  receiving  computer  to  determine  whether  the  packet  was 
garbled  during  transmission;  then  the  information  field  containing  the  data,  message,  or  message  fragment  (e.g.,  a  virtual  tank’s  position),  which 
the  rest  of  the  packet  merely  transports;  and  finally  a  flag  marking  the  end  of  tlie  packet. 

'^^Aframe  is  the  information  represented  by  apacket.  A  frame  is  a  logical  object;  apacket  is  aphysical  (if  intangible)  ob]&ct.  A  datagram  is  a 
frame  used  in  certain  types  of  networking  protocols  (standard  communication  procedures),  such  as  tlie  Transmission  Control  Protocol  (TCP) 
and  Internet  Protocol  (IP). 

In  the  development  of  electronic  equipment,  a  breadboard  version  refers  to  a  prototype  tliat  performs  the  electronic  functions  required  but 
is  not  constructed  mechanically  like  the  intended  product.  In  the  early  days  of  radio  development,  experimental  or  hobbyist  radio  equipment 
was  often  constructed  by  mounting  components  on  a  breadboard  borrowed  from  tlie  experimenter’s  kitchen  (or  on  a  wooden  board  of  similar 
shape),  instead  of  a  more  costly  steel  or  aluminum  chassis  of  tlie  type  used  for  commercial  and  military  equipment. 
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of  long-haul  networking  the  next  month  was  suc¬ 
cessful.  By  the  fall  of  1989,  six  SIMNET  sites  or 
nodes  had  been  established  at  Fort  Knox;  Fort 
Rucker,  Alabama;  Fort  Benning,  Georgia;  and  Eu¬ 
ropean  sites  (2). 

At  these  sites,  four-man  tank  crews  took  their 
stations  inside  simulators  built  to  look  like  tanks 
on  the  inside  (as  shown  in  figure  1-1).  The  crew 
members  operated  switches,  knobs,  levers,  and 
other  controls  replicating  the  controls  in  a  real 
tank  of  the  type  being  simulated.  Through  the  sim¬ 
ulated  periscopes  and  gunsight,  they  saw  scenery, 
including  simulated  vehicles,  pass  by  as  they 
drove  or  traversed  the  turret. 

Each  simulator  simulated  the  dynamics  of  a 
tank’s  engine,  transmission,  drive  train,  and  sus¬ 
pension:  the  scenery  would  pass  more  slowly 
when  the  simulator  ascended  a  hill  (because 
tracked  vehicles  slow  down  on  a  grade)  or  tra¬ 
versed  a  muddy  field  (because  tracked  vehicles 
slip  or  bog  down  in  deep  mud).  In  addition,  some¬ 
times  the  gun  or  engine  would  fail  to  work,  be¬ 
cause  unreliability  was  simulated,  as  was  damage 
to  electrical,  hydraulic,  weapons,  and  other  sys¬ 
tems.  The  virtual  vehicles  broke  down,  consumed 
fuel,  and  expended  ammunition  at  rates  calculated 
from  experience  with  real  vehicles. 

I  Computer  image  Generation 

SIMNET  gambled  on,  and  successfully  exploited, 
breakthroughs  in  computer  image  generation.  In 
the  1960s,  DARPA  sponsored  head-mounted  dis¬ 
play  research  that  would  find  application  in  future 
flight  simulators  (147).  In  the  1970s  high-resolu¬ 
tion  CIGs  in  flight  simulators  projected  wide- 
angle  views  that  followed  the  movement  of  the 
pilot’s  eyes  on  a  dome  above  the  pilot’s  head. 
Computer-generated  imagery  was  used  in  the  Ad¬ 
vanced  Simulator  for  Pilot  Training  (ASPT)  in 
1976;  in  the  Visual  Technology  Research  Simula¬ 


tor  at  the  Naval  Training  Systems  Center  (NTSC) 
in  Orlando,  Horida  (now  the  Naval  Air  Warfare 
Center  Training  Systems  Division),  and  in  the 
Unit  Conduct  of  Fire  Trainer  (UCOFT),  a  full- 
mission  tank  simulator  (2). 

The  TGTs  used  in  the  TTGT  were  simulators 
with  a  handle  with  triggers  on  the  front  and  an  eye¬ 
piece  through  which  to  view  scenes  played  back 
from  a  video  recording.  The  five  trainees  in  the 
TTGT  would  be  shown  the  same  video-disc-gen¬ 
erated  scene  and  compete  to  be  the  first  to  fire  on 
an  enemy  tank  (18).  Random  access  video,  in 
which  the  trainee  could  change  direction  and  get  a 
different  view  (provided  it  had  been  prerecorded), 
was  the  next  step.  But  even  random  access  video 
disc  technology,  in  its  infancy  in  1979  (18),  was 
not  flexible  enough  for  interactive  training.  All 
the  possible  images  could  not  be  recorded  in  ad¬ 
vance  on  video.  They  had  to  be  generated  as  need¬ 
ed  by  the  computer.  CIG  seemed  integral  to 
accomplishing  interactive  simulation. 

However,  during  the  late  1970s  CIG  cost  $2 
million  to  $3  million  per  channel,^^  and  CIG- 
based  trainers  ranged  from  $5  million  for  UCOFT 
to  $30  million  for  ASPT  (2).  For  SIMNET  simu¬ 
lators,  the  concept  was  to  have  the  simulator’s  mi¬ 
croprocessors  project  the  appropriate  moving 
image  for  each  member  of  a  tank  crew.  This  re¬ 
quired  eight  different  views  or  CIG  channels  for 
each  SIMNET  tank  simulator.  Lower  cost  became 
imperative. 

By  the  end  of  the  SIMNET  project,  the  cost  of 
each  visual  display  subsystem  was  less  than 
$100,000  (2).  The  20-  to  30-fold  cost  reduction 
from  the  cost  per  CIG  channel  in  the  early  1970s  is 
partly  attributable  to  the  decline  in  cost  and  the  in¬ 
crease  in  scale  (i.e.,  number  of  transistors)  of  inte¬ 
grated  circuits  dur  ing  that  period.  It  is  also  partly 
attributable  to  the  fact  that  each  display  system 
(periscope  or  gunsight  simulator,  called  a  vision 


^  “Channel”  or  “vi.svial  channel”  when  used  with  respect  to  [computer-generated  imagery]  systems,  implies  one  computing  entity,  often 
part  of  several,  which  processes  image  data  for  that  part  of  the  simulation  presented  visually  tlirough  a  display  system.  Such  parts  may  be  one  of 
several  elements  of  the  outside  visual  scene. . .  the  greater  tlie  number  of. , ,  channels,  the  larger  tlie  potential. . .  [Held  of  view]  provided  by  the 
display  system”  (136). 
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block)  for  which  a  SIMNET  CIG  generated  imag¬ 
ery  displayed  fewer  pixels^^  than  did  a  typical 
flight  simulator  display.  This  could  be  justifed  be¬ 
cause  a  tank  crewman  looking  through  a  periscope 
has  a  smaller  field  of  view  than  does  a  fighter  pilot 
looking  through  a  canopy.  (A  SIMNET  tank  simu¬ 
lator  did  not  provide  an  open-hatch  view  for  the 
commander.) 

Today,  as  competition  continues  to  drive  the 
costs  of  computer  image  generation  down,  the 
compromise  on  resolution  is  being  reversed:  new¬ 
er  simulations  are  using  increasingly  higher  reso¬ 
lution  graphics  to  serve  other  uses  besides 
training.  A  1992  Defense  Science  Board  summer 
study  of  simulation,  readiness,  and  prototyping 
concluded  that  CIG  technology  advances  would 
be  driven  by  the  commercial  sector  in  the  near  fu¬ 
ture  (191).  Performance  and  resolution  of  high- 
end  graphics  work  stations  are  increasing,  and 
cost  is  decreasing.  Advances  by  the  consumer 
electronics  industry  and  video  game  producers  are 
creating  widespread  benefits  for  the  visual  display 
market,  including  better  and  cheaper  image  pro¬ 
jection  systems  and  liquid  crystal  displays,  which 
are  used  in  head-mounted  displays  for  flight  simu¬ 
lation,  surgery,  and  automotive  repair,  and  in  in¬ 
frared  scopes  such  as  those  used  in  the  Persian 
Gulf  War  (33). 

The  fidelity  of  simulated  scenes  depends  not 
only  on  the  types  of  phenomena  modeled  but  also 
on  scene  complexity  and  display  resolution.  Most 
image  generators  represent  surfaces  (e.g.,  terrain) 
as  connected  polygons  (e.g.,  triangles),  so  the 
number  of  polygons  in  a  scene  is  a  measure  of 
scene  complexity.  Rendering  a  more  complex 
scene  takes  longer,  so  a  more  useful  performance 
metric  is  polygons  per  second,  calculated  by  mul¬ 


tiplying  the  scene  complexity  in  polygons  by  the 
frame  rate  in  Hz  (147). 

The  image  generator  used  in  the  developmental 
SIMNET  Ml  tank  simulator  could  render  scenes 
at  120  thousand  polygons  per  second^^ — thus  it 
was  named  the  120T  CIG  (98).  Its  frame  rate  was 
15  Hz  (2),  so  it  rendered  scenes  composed  of 
8,000  polygons.  This  composite  was  divided 
among  eight  channels — one  for  each  vision  block. 
There  were  three  vision  blocks  for  the  driver,  three 
for  the  commander,  and  one  for  the  loader,  each 
displaying  an  image  320  pixels  wide  by  128  pixel 
high,  and  a  higher-resolution  vision  block  for  the 
gunner  (98). 

Production  models  of  SIMNET  simulators 
used  GTl  00- series  CIGs  produced  by  BBN  Delta 
Graphics,  as  was  the  120T.  The  GTIOO  series  in¬ 
cluded  the  GTlOl,  which  was  equivalent  to  the 
120T;  the  GTl 02,  which  was  equivalent  to  two 
1 20Ts  and  was  usable  for  1 6  channels  or  for  eight 
higher-resolution  channels;  the  GTl  10,  which 
was  equivalent  to  10  GTlOls;  and  the  GT120, 
which  was  equivalent  to  two  GTl  10s  (98). 

Each  CIG  had  to  store  an  object  representa¬ 
tion — ^information  about  the  shape,  size,  texture, 
and  pigmentation — of  each  type  of  simulated  ve¬ 
hicle  participating  in  the  simulation;  it  also  had  to 
store  representations  of  the  terrain,  vegetation, 
and  cultural  features  (buildings,  roads,  and 
bridges,  etc.).  Terrain  elevation  information  was 
organized  into  load  modules  each  representing  a 
500-mi2  patch  of  terrain  as  a  surface  composed  of 
32  triangles  (which  are  polygons).  Each  CIG 
could  store  196  load  modules  at  one  time  in 
memory — enough  to  represent  a  square  3.5  km  on 
a  side.  Each  simulator  was  required  to  display  ob- 


Pixel  (or  pel)  is  an  abbreviation  for  picture  element — a  dot  on  a  display  screen.  For  comparison,  a  Super  V  QA  (SVGA)  computer  monitor 
caa  display  an  image  1,024  pixels  wide  by  768  pixels  high. 

For  comparison,  the  current  state  of  the  art  in  commoditized  systems  is  illustrated  by  tlie  capabilities  of  tlie  fully  scaled-up  Silicon  Graph¬ 
ics,  Inc.,  Onyx  Reality  Engine  2  (over  2  million  triangles  per  second  ( 129))  and  of  tlie  Division,  Inc. ,  Pixel  Planes  6  system  (4  million  photo-tex¬ 
tured  (147)  or  5  million  Gouraud-shaded  (147)  triangles  per  second  (28)).  Considerable  scaling  will  he  required  to  meet  tlie  rendering  require¬ 
ments  that  ARPA  has  specified  for  its  Simulation-Based  Design  (SBD)  project:  30  million  polygons  at  30  frames  per  second,  or  300  polygons 
per  second  (147). 
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jects  and  terrain  out  to  a  range  of  3  km,  so  when  a 
simulated  vehicle  moved  500  m  in  one  direction, 
it  would  load  into  memory  the  load  modules  rep¬ 
resenting  the  next  row  of  500  mi^  terrain  patches 
beyond  those  already  represented  ahead  of  it, 
overwriting  the  load  modules  representing  the 
row  of  500  mi^  square  terrain  patches  that  it  was 
leaving  behind. 

Typically  the  heights  of  the  vertices  of  the  trian¬ 
gular  patches  of  terrain  were  elevations  at  125-m 
intervals.  Representing  coastlines  and  rivers  in  a 
recognizable  form  required  using  elevations  at 
smaller  intervals,  which  the  CIG  could  accommo¬ 
date  as  long  as  each  load  module  contained  no 
more  than  32  triangles.  A  terrain  patch  that  in¬ 
cluded  a  coastline  might  use  most  of  its  32 
triangles  to  represent  the  coastline  and  would  need 
few  to  represent  the  ocean. 

I  Synthetic  Environments 
Terrain  Database  Construction 

Battlefield  interaction  takes  place  on  complex  ter¬ 
rain,  and  the  virtual  terrain  of  a  synthetic  environ¬ 
ment  should  reflect  its  important  features.  A  tank 
commander’s  view  of  the  battlefield  is  very  differ¬ 
ent  from  that  of  a  pilot,  and  tank  commander’s  re¬ 
quirements  for  detail  in  terrain  representation  are 
different.  Having  a  full  field  of  view  is  critical  for 
pilots;  providing  it  requires  a  terrain  database  and 
a  means  of  rendering  and  displaying  the  scene  that 
the  pilot  would  see,  whatever  the  direction  of  his 
gaze.  A  tank  driver,  on  the  other  hand,  needs  only 
the  restricted  field  of  view  provided  by  his  vision 
block;  however,  his  tank  simulator  needs  to  know 
soil  properties  that  affect  traction.  Ideally,  the  be¬ 
havior  of  the  soil  should  also  be  simulated.  For  ex¬ 
ample,  if  wet  or  sandy,  the  soil  should  develop  ruts 
if  many  tracked  vehicles  traverse  it. 

A  significant  indirect  cost  of  the  use  of  comput¬ 
er  image  generation  is  the  digitization  of  terrain 
data  for  use  as  terrain  databases.  Those  databases 
contain  all  the  information  about  a  piece  of  land 
necessary  for  the  computer  to  project  hills,  trees, 
and  other  features  combatants  would  need  to  see. 
A  terrain  database  describing  Fort  Hunter  Liggett, 


California,  has  been  widely  used  for  DIS  demon¬ 
strations  because  of  its  availability  and  detail. 

Rapid  construction  of  terrain  databases  is  need¬ 
ed  for  mission  rehearsal  on  a  contingency  basis  for 
two  DoD  programs  in  particular:  the  Advanced 
Research  Projects  Agency’s  Warbreaker  program 
(see  box  3)  and  the  Air  Force’s  Special  Operation¬ 
al  Forces  Aircrew  Training  System.  The  Air  Force 
requires  the  capability  to  rehearse,  in  simulation, 
for  a  mission  anywhere  in  the  world  with  only  48 
hours  advance  notice,  using  a  site-specific  (not 
just  typical)  terrain  database  representing  500,000 
nmP  with  photographic  texture  in  multiple  levels 
of  detail  (89).  DoD’s  draft  MSMP  contains  a  num¬ 
ber  of  goals  for  rapid  terrain  database  generation, 
including  a  requirement  to  “demonstrate  an  initial 
capability  in  FY  1996  to  produce,  within  one 
week,  standard  terrain  data  to  meet  M&S  func¬ 
tional  area  requirements  contained  within  a  nomi¬ 
nal  2,500  km^  area,’’  and  a  requirement  to 
“demonstrate  in  FY  1997  the  capability  to  pro¬ 
duce  standard  terrain  data  to  meet  modeling  and 
simulation  functional  area  requirements  con¬ 
tained  within  a  nominal  2,500  km^  area  (with 
three-dimensional  terrain,  including  three-dimen¬ 
sional  man-made  features,  reasonably  attributed), 
within  72  hours’’  (195). 

The  Undersecretary  of  Defence  for  Acquisition 
and  Technology  has  recently  appointed  the  De¬ 
fense  Mapping  Agency’s  Teirain  Modeling  Proj¬ 
ect  Office  to  be  the  modeling  and  simulation 
Executive  Agent  for  terrain  modeling,  to  be  re¬ 
sponsible  for  meeting  the  requirements  for  terrain 
database  generation  set  forth  in  the  MSMP. 

Cartography,  the  depiction  of  the  three-dimen¬ 
sional  Earth  in  two-dimensional  media,  has  been 
revolutionized  by  the  use  of  airborne  and  space- 
based  remote  sensing  imagery.  Remotely  sensed 
data,  unlike  ground  survey  data,  provides  multidi¬ 
mensional  views  of  terrain  using  multispectral 
scanners,  infrared  cameras,  imaging  radars 
(36,198),  and  photography.  Another  revolution 
has  been  in  the  direct  use  of  digital  teirain.  The  ul¬ 
timate  product  of  cartographic  data  need  not  nec¬ 
essarily  be  a  paper  map.  The  raw  electronic  digital 
cartographic  information  can  be  directly  used  and 
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BOX  3:  Warbreaker 


The  Advanced  Research  Projects  Agency’s  Warbreaker  program  focuses  on  finding  and  destroy¬ 
ing  time-critical  targets,  including  tactical  ballistic  missile  launchers,  such  as  those  that  launched  Scud 
missiles  during  the  Persian  Gulf  war.  The  program  is  a  part  of  DoD’s  Precision  Strike  Initiative,  Like  the 
Air  Force’s  Semi-Automatic  Ground  Environment  air  defense  system'and  some  subsequent  projects,^ 
Warbreaker  uses  DIS  to  simulate  a  complex  system  of  systems:  the  weapons,  sensor,  intelligence,  and 
vehicle  systems  that  must  work  together  to  destroy  critical  mobile  targets,  The  first  step  of  the  program 
was  to  recreate  in  simulation  the  systems  and  vehicles  involved  in  targeting  Scud  missiles,  modeling 
their  limitations  as  well  as  their  capabilities.  The  next  step  is  to  improve  the  process  for  a  more  success¬ 
ful  outcome. 

The  objective  of  the  program  is  to  change  system  design,  acquisition,  and  training  in  this  particu¬ 
lar  area  by  evaluating  a  mix  of  existing  and  potential  systems  to  determine  how  they  can  operate  to¬ 
gether  more  affordable  and  efficiently  and  to  identify  new  technologies  needed  to  accomplish  the  task. 
The  technologies  that  will  be  developed  to  accomplish  the  mobile  targeting  include  end-to-end  integra¬ 
tion  of  data  collection,  targeting  technologies,  and  allocation  of  resources  for  strike.'Technologies  that 
will  be  exploited  include  those  of  data  fusion,  battle  management,  and  intelligence  collection,  proces¬ 
sing  and  distributing.  Connecting  national  assets  to  theater-level  targeting  systems  that  support  mobile 
and  fixed  target  acquisition  is  a  part  of  the  design. 

The  program  is  part  of  a  Distributed  Defense  Simulation  testbed'that  serves  several  larger  ob¬ 
jectives  as  well.  ARPA  hopes  to  use  the  Warbreaker  infrastructure  as  a  procurement  tool  that  brings 
together  the  technologist,  the  developer,  and  the  user  for  increased  effectiveness  in  weapon  systems 
development  and  procurement,  Warbreaker  will  also  establish  a  platform  for  integrating  and  networking 
existing  systems  more  effectively.  Ultimately,  the  goal  is  to  establish  a  DoD-wide  multilevel  secure  DIS 
infrastructure  in  which  diverse  simulations  and  simulators  can  be  networked  into  a  scenario. 


'J.F.  Jacobs,  The  SAGE  Air  Defense  Systems:  A  Personal  History  (Bedford,  MA:  MITRE  Corp.,  1986) 

'An  example  is  the  simulated  interoperation  of  the  U.S.  Air  Force’s  Tactical  Air  Control  System  and  U.S.  tactical  air  defense 
systems  with  air  defense  systems  of  the  North  Atlantic  Treaty  Organization.  See  J.E.  Freedman  and  S.H.  Starr,  “Use  of  Simulation  in  the 
Evaluation  of  the  IFFN  Process,  ”  in  North  Atlantic  Treaty  Organization  Advisory  Group  for  Aerospace  Research  and  Development 
Conference  Proceedings  No.  268  (AGARD-CP-268),  presented  at  a  Meeting  of  the  Avionics  Panel,  Paris,  France,  15-19  October 
1979. 

*U.S.  Department  of  Defense,  Defense  Advanced  Research  Projects  Agency,  "Warbreaker  (Critical  Mobile  Targets)  System 
Engineering  and  Integration  Research  and  Development,”  preproposal  conference  presentation,  Arlington,  VA,  Apr.  6,  1992). 

^U.S.  Department  of  Defense,  Defense  Advanced  Research  Projects  Agency,  Advanced  Systems  Technology  Office,  "Dis¬ 
tributed  Defense  Simulation  for  System  Engineering  In  the  Acquisition  Process:  The  SERCES  and  the  Warbreaker  Programs,"  industry 
briefing  paper  (Arlington,  VA:  1992). 

SOURCE:  Office  of  Technology  Assessment,  1995, 


imbedded  within  other  systems  (96).  Direct  digi¬ 
tal  data  are  used  for  navigation  and  routing,  but  for 
many  uses  a  map  is  more  informative  than  any  pic¬ 
ture  could  be. 

Three-dimensional  terrain  databases  have  been 
generated  from  photographs  in  two  weeks  using 
four  weeks  of  manpower.  Overhead  imagery  can 
be  turned  into  three-dimensional  scenes  showing 
key  buildings,  topography,  and  important  features 


for  military  operations  including  telephone  poles, 
power  lines,  and  roads  (37).  Digital  information 
on  troubled  areas  is  increasing,  and  techniques  for 
direct  use  of  remotely  sensed  data  are  improving 
(97).  The  final  step  will  be  to  learn  to  feed  that  in¬ 
formation  directly  into  targeting  weapon  systems. 

Challenges  for  terrain  database  generation  in¬ 
clude  the  need  to  combine  data  collected  by  a  vari¬ 
ety  of  sensors  and  stored  on  various  media  in  a 
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variety  of  formats  and  security  classifications 
(146).  Timely  collection  of  terrain  data  is  also  a 
challenge.  One  of  many  technologies  that  might 
be  useful  is  airborne  or  spacebome  interferometric 
synthetic-aperture  radar,  which  can  collect  eleva¬ 
tion  data  even  at  night  and  through  cloud  cover 
(36,198).  A  different  type  of  challenge  is  updating 
terrain  database  in  real  time  to  represent  changes 
in  the  terrain,  such  as  cratering,  that  should  occur 
as  a  consequence  of  simulated  actions,  such  as 
barrage  or  bombardment  (147). 

Although  technological  advance  is  increasing 
scene  rendering  detail  and  speed,  it  does  not  yet 
satisfy  the  most  demanding  requirements  for  ter¬ 
rain  detail  and  extent,  such  as  for  strike  mission 
simulation  and  simulation-based  design  of  com¬ 
plex  systems  such  as  ships  (147).  This  is  a  critical 
issue  in  real-time  interactive  simulation  as  is  ter¬ 
rain  database  correlation  (agreement  of  the  data¬ 
bases,  describing  the  same  real  or  synthetic 
terrain)  among  different  simulators.  Terrain  data¬ 
base  correlation  problems  are  as  old  as  DIS  (in  the 
generic  sense  of  the  term),  having  occurred  in  the 
development  of  the  Air  Force’s  Semi-Automatic 
Ground  Environment  air  defense  system  (76)  and 
in  numerous  subsequent  cases. 

Other  Synthetic  Environments 

The  draft  MSMP  includes  action  items  intended 
to  fulfill  the  objective  of  providing  timely  and  au¬ 
thoritative  representations  of  the  natural  environ¬ 
ment — ^including  the  oceans,  the  atmosphere,  and 
space,  as  well  as  terrain.  Modeling  these  environ¬ 
ments  was  not  essential  in  the  early  days  of  SIM- 
NET,  when  it  was  sufficient  to  fight  simulated 
tank  battles  at  high  noon  on  clear  days.  Expansion 
of  DIS  to  accomodate  joint  exercises  involving  a 
greater  variety  of  platforms  and  sensors  (acoustic, 
radio,  infrared,  and  optical)  set  in  various  climates 
and  times  of  day  has  made  it  imperative  to  model 
many  aspects  of  the  environment — ^not  just  ter¬ 
rain.  The  Undersecretary  of  Defense  for  Acquisi¬ 
tion  and  Technology  has  recently  appointed 
Executive  Agents  for  modeling  the  oceans,  the 
lower  atmosphere,  the  upper  atmosphere,  and  out¬ 
er  space,  to  be  responsible  for  meeting  the  require¬ 


ments  for  terrain  database  generation  set  forth  in 
the  MSMP. 

I  Semiautomated  Forces 

Realistic  collective  training  of  an  Army  unit  re¬ 
quires  simulation  of  some  situations  in  which  the 
unit  is  greatly  outnumbered  by  opposing  forces.  It 
would  be  expensive  if  all  the  opposing  forces  were 
virtual  forces — soldiers,  perhaps  experts  in  for¬ 
eign  force  doctrine,  operating  simulators.  To  avert 
this  cost,  DoD  has  developed,  and  is  improving, 
automated  computer-generated  forces  (CGF)  and 
semiautomated  forces.  Using  automated  or  semi¬ 
automated  opposing  forces  allows  a  simulation  to 
populate  the  virtual  battlefield  with  a  larger  num¬ 
ber  of  enemy  troops  (and  perhaps  neutrals  and 
noncombatants)  and  to  test  proposed  doctrine  and 
tactics  on  a  larger  scale.  SAP  were  required  to  sim¬ 
ulate  large-scale  battle  exercises  for  command 
training,  such  as  REFORGER  (see  box  4).  How¬ 
ever,  the  effectiveness  of  using  an  automated 
force,  is  limited  by  the  realism  of  its  behavior.  The 
original  SAF  simulation  in  SIMNET  was  rudi¬ 
mentary,  but  the  technology  is  being  improved  by 
efforts  such  as  the  Advanced  Research  Projects 
Agency’s  Intelligent,  Imaginative,  Innovative, 
Interactive  What  If  Simulation  System  for 
Advanced  Research  and  Development  (I^ 
WISSARD)  program. 

Current  problems  and  limitations  in  semiauto¬ 
mated  forces  include  the  number  of  control  per¬ 
sonnel  required  to  manage  SAF  in  an  exercise  and 
the  realism  of  SAF  behavior.  The  goal  is  SAF  that 
are  so  capable  that  the  operator  of  a  simulator  will 
be  unable  to  tell  the  difference  between  SAF  and 
forces  controlled  by  other  manned  simulators. 
Different  types  of  entities  (tanks,  fighter  planes, 
and  dismounted  soldiers,  for  example)  pose  dif¬ 
ferent  challenges  for  simulation. 

Efforts  towards  this  goal  are  resulting  in  a  shift 
from  SAF  that  are  algorithm-based  (as  in  SIM¬ 
NET)  to  SAF  that  ai'e  based  on  AI  (artificial  intel¬ 
ligence).  In  the  original  algorithm-based  SAF,  the 
computer  would  move  each  vehicle  step  by  step 
based  on  its  next  objective.  Each  tank  stays  in 
formation  with  other  tanks  and  avoids  obstacles. 
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BOX  4:  REFORGER 


Milestones  in  distributed  simulation  were  achieved  during  the  1992  REFORGER  exercise.  In  pre¬ 
vious  years  the  annual  exercise  had  deployed  thousands  of  troops  in  Europe.  In  1992  much  of  the  ac¬ 
tion  was  simulated  by  networking  and  interoperating  the  Air  Force’s  Air  Warfare  Simulation  and  the 
Army’s  Corps  Battle  Simulation  using  a  new  translation  protocol,  the  Aggregate  Level  Simulation  Proto¬ 
col.  The  Aggregate  Level  Simulation  Protocol  synchronized  the  timing  of  events  in  the  two  constructive 
simulations  and  relayed  event  and  status  messages  from  each  to  the  other,  so  that  the  results  of  one 
could  influence  the  other,  ’ 

Perhaps  the  most  significant  feature  of  the  exercise,  however,  was  the  saving  of  $34  million  over 
REFORGER  ’88.  REFORGER  ’88  had  deployed  17,500  people;  REFORGER  ’92  deployed  only  6,500 
people,  but  simulated  the  actions  of  175,000  people,  including  the  semiautomated  forces  controlled  by 
200  experts  at  Fort  Leavenworth,  Kansas.  The  post-exercise  review  concluded  that  the  training  accom¬ 
plished  in  REFORGER  ’88  was  negative  for  lower-level  troops, "who  mostly  sat  at  roadsides,  and  mini¬ 
mal  for  command  staff.  REFORGER  ’92  had  fully  documented,  free-play  interaction  resulting  in  positive 
training  at  all  levels.  However,  because  REFORGER  ’92  was  live  at  the  command  post  level  but 
constructive  at  the  soldier  level,  fewer  soldiers  were  trained  than  In  1988,  This  Is  an  improvement  only  if 
one  accepts  the  proposition  that  the  negative  learning  by  soldiers  in  REFORGER  ’88  outweighed  the 
value  of  any  positive  learning  at  that  echelon.  Another  way  to  summarize  the  results:  REFORGER  ’92 
demonstrated  the  potential  of  constructive  simulation  to  save  money  and  other  resources  in  providing 
realistic  command  and  staff  training  in  the  context  of  a  large-scale  scenario,  but  it  was  not  designed  to 
provide  realistic  training  opportunities  for  individual  soldiers. 


'Earlier  in  1992,  in  the  annual  Ulchi  Focus  Lens  exercise,  the  Aggregate  Level  Simulation  Protocol  had  linked  computers 
mnning  the  Corps  Battle  Simulation  In  Korea  with  computers  mnning  the  Air  Warfare  Simulation  and  the  Navy's  Research,  Evaluation 
and  Systems  Analysis  model  at  the  Warner  Preparation  Center  in  Germany.  Later,  a  constructive  Marine  Corps  simulation  was  added 
to  the  confederation  of  simulations  that  could  be  linked  by  the  Aggregate  Level  Simulation  Protocol.  See  MiTRE  Corporation,  Aggre¬ 
gate  Level  Simulation  Protocol  (ALSP)  Program  Status  and H/sfo/y (McLean,  VA:  March  1993);  available  online  at  <URL:http://alsp.ar- 
pa.mil/aIsp/89-92-history/89-92-history.html>. 

*U.S.  Department  of  Defense,  Office  of  the  Undersecretary  of  Defense  for  Acqusition,  Defense  Science  Board  Task  Force  on 
Simulation,  Readiness,  and  Prototyping,  Impact  of  Advanced  Distributed  Simulation  on  Readiness,  Training  and  Prototyping,  con¬ 
densed  (Washington,  DC:  January  1993). 


If  the  lead  tank  sees  an  obstacle  and  changes  direc¬ 
tion,  the  next  tank  may  slowdown  to  avoid  a  colli¬ 
sion.  As  the  lead  tank  moves  around  the  obstacle 
and  gets  back  on  course,  the  second  tank  has  to 
readjust  to  get  back  in  formation.  Multiply  these 
decisions  by  16  tanks  and  the  result  is  sometimes 
erratic  behavior. 

This  control  base  has  been  improved  with  a 
concurrent  control  base  that  takes  the  same  goals 
and  arbitrates  them  using  weighted  distributions 
for  each  goal  in  an  ongoing  balancing  of  the  direc¬ 
tive  for  the  tank.  Using  this  “fuzzy  logic”  meth¬ 
od,  also,  additional  objectives  for  a  mission 


scenario  can  be  added  to  the  code  for  a  more  com¬ 
plex  response. 

The  next  step  up  in  complexity  is  knowledge 
acquisition  by  the  SAP  using  techniques  of  artifi¬ 
cial  intelligence,  including  expert  systems  and 
case-based  reasoning.  Expert  systems  are  devel¬ 
oped  by  knowledge  engineering:  interviewing  an 
expert  (e.g.,  a  pilot)  to  discover  the  rules  of  thumb 
and  information  he  or  she  uses  to  solve  a  problem, 
then  putting  that  information  into  the  software  in  a 
form  that  a  computer  can  use  to  direct  action.  This 
is  a  laborious  process.  An  alternative  is  case-based 
reasoning:  designing  SAP  software  that  can  learn 
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BOX  4:  REFORGER  (Cont’d) 

REFORGER  *88 

REFORGER  *92 

Deployed  troops 

17,487 

6,500 

Engaged  troops 

more  than  97,000 

more  than  20,000 

Engaged  tracked  vehicles 

7,000 

135 

Engaged  tanks 

1,080 

0 

Simulated  troops 

more  than  97,000 

175,000 

Simulated  tracked  vehicles 

7,000 

12,810 

Simulated  tanks 

1,080 

1,950 

Maneuver  HQ 

35 

41 

Exercise  scale 

corps  vs.  corps 

army  group  vs.  front 

Style 

umpire 

free  play,  intelligent  opposing  force 

Intelligence  play 

scripted 

sensor  level  feedback 

Training  effect  (battalion  and  below) 

negative  (on  roads) 

positive  (at  home  station) 

After-action  review 

anecdotal 

analytic  data 

cost 

$53.9  million 

$19,5  million 

SOURCE:  Office  of  Technology  Assessment,  1995. 

from  its  own  successes  and  failures  in  simulated 
combat.  For  the  computer  to  “learn”  from  experi¬ 
ence,  it  must  be  able  to  acquire  knowledge  from 
new  experiences  in  real  time  and  add  it  to  its 
knowledge  base  (i.e.,  expertise)  in  such  as  way  as 
to  be  able  to  use  it  later.  The  computer’s  knowl¬ 
edge  base  consists  of  many  cases  that  the  comput¬ 
er  uses  in  real  time  to  match  the  scenario  it  is 
dealing  with  and  respond  accordingly.  If  the  com¬ 
puter  sees  something  new,  it  is  added  to  the  data¬ 
base.  Case-based  reasoning  is  the  basis  for 
machine  learning  in  l^WISSARD  (142). 

Currently,  such  added  case-based  learning  is 
coded  in  by  hand  after  an  after-action  review. 
Ideally,  a  computer  could  do  such  a  review  auto¬ 
matically  in  realtime.  A  pilot  could  train  the  com¬ 
puter  by  dogfighting  with  it.  If  the  pilot  performed 
a  maneuver  that  the  machine  had  never  seen  be¬ 
fore,  the  computer  might  be  killed  but  would  add 
that  trick  to  its  knowledge  base.  If  the  pilot  tried  to 
repeat  his  success,  he  would  find  that  the  comput¬ 
er  does  not  respond  in  the  same  way.  The  scenario 


would  turn  out  differently.  This  could  provide  a 
richer  training  experience  for  the  pilot  and  make 
the  simulator  training  useful  longer. 

I  Simulation  Architecture 

The  DoD’s  draft  Modeling  and  Simulation  Master 
Plan  defines  architecture  as  “the  structure  of  com¬ 
ponents  in  a  program  [or]  system,  their  inter¬ 
relationships,  and  principles  and  guidehnes 
governing  their  design  and  evolution  over  time” 
(195).  It  calls  for  the  development  of  a  compre¬ 
hensive,  abstract  high-level  architecture  for  DoD 
simulation:  “the  major  functional  elements,  inter¬ 
faces,  and  design  rules,  pertaining  as  feasible  to  all 
DoD  simulation  applications,  and  providing  a 
common  framework  within  which  specific  system 
architectures  can  be  defined”  (75,167).*' The  cur¬ 
rent,  still  evolving  DIS  architecture  evolved  from 
that  of  SIMNET. 

When  the  SIMNET  program  began,  most  sim¬ 
ulation  data  processing  was  done  by  a  central 


"Examples  of  specific  system  architectures  are  the  proposed  Communications  Architecture  for  Distributed  Interactive  Simulation  and  the 
object  and  event  architectures  embodied  in  the  Distributed  Interactive  Simulation  protocols. 
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computer.^^  Initially  SIMNET  was  envisioned  to 
use  a  similar  architecture:  control  inputs  from  sev¬ 
eral  simulators  would  be  transmitted  to  a  central 
computer,  which  would  process  the  data  and  send 
video  signals  back  to  display  systems — television 
receivers — ^in  all  the  simulators.  Each  simulator 
would  require  several  television  channels — one 
per  simulated  periscope  or  gunsight — and  it  was 
soon  realized  that  a  cable  from  the  central  comput¬ 
er  would  only  be  able  to  serve  a  few  simulators. 
The  architecture  was  not  scalable — that  is,  the 
number  of  networked  simulators  could  be  in¬ 
creased  eonomically. 

SIMNET’s  program  manager.  Jack  Thorpe,  in¬ 
vented  a  different,  more  scalable  architecture. 
Each  simulator  would  be  designed  to  do  its  own 
simulation  processing  and  image  generation. 
Each  -simulator  was  to  be  a  self-contained  unit, 
with  its  own  host  microprocessor,  terrain  data¬ 
base,  display  and  sound  systems,  cockpit  controls, 
and  network  interface.  Each  would  generate  all  the 
sights  and  sounds  necessary  to  train  its  crew.  Each 
tank  crew  member  would  see  a  part  of  the  virtual 
world  created  by  the  graphics  generator  using  the 
terrain  data  base  and  information  arriving  via  the 
net  regarding  the  movements  and  status  of  other 
simulated  vehicles  and  battle  effects — the  precise 
part  being  defined  by  the  crew  member’s  line  of 
sight:  forward  for  the  tank  driver  or  from  any  of 
three  viewing  ports  in  a  rotatable  turret  for  the  tank 
commander.^^ 

With  the  shift  from  centralized  processing  to 
autonomous,  distributed  processing,  the  concept 
of  data  communication  changed.  Each  simulator 
was  to  communicate  with  every  other  simulator 
instead  of  with  a  central  mainframe  computer.^^  In 
addition,  each  simulator  is  responsible  for  what  it 
represents  on  its  displays.  Each  simulator’s  mi¬ 
croprocessor  carries  a  model  for  all  of  the  objects 


on  the  network,  including  itself,  that  describes 
that  object,  its  state,  and  activity  and  is  responsible 
for  updating  that  model  when  something  changes. 

In  order  to  overcome  the  problem  of  massive 
amounts  of  data  exchange,  the  SIMNET  team  de¬ 
cided  that  constant  updates  were  not  necessary, 
only  significant  changes  needed  to  be  communi¬ 
cated.  In  between,  the  simulator  projects  other  ob¬ 
jects,  based  on  its  internal  model  and  assuming 
there  is  no  change,  according  to  its  last  informa¬ 
tion:  if  a  tank  is  traveling  a  particular  direction  at  a 
certain  speed,  a  SIMNET  simulator  projects  the 
vehicle  continuing  on  in  that  direction  at  that 
speed  until  told  otherwise.  This  way,  the  action 
continues  between  update  information  and  the 
speed  of  update  merely  influences  the  smoothness 
of  the  transition  to  a  new  state.  A  more  sophisti¬ 
cated  version  of  this  “dead  reckoning”  idea  is 
“remote  entity  approximation,”  which  incorpo¬ 
rates  not  only  first  order  approximations  like  ve¬ 
locity,  but  also  second  order  approximations  like 
acceleration.  The  key  idea  behind  both  dead  reck¬ 
oning  and  remote  entity  approximation  is  the  sub¬ 
stitution  of  local  data  processing,  which  is  easy, 
for  data  transmission  to  each  simulator  over  the 
network,  which  is  expensive  and  constrained. 

The  basic  design  of  the  network  for  SIMNET 
was  set  by  1985.  Simulators  were  linked  through 
local  area  networks  onto  a  larger  wide  area  net,  the 
long-haul  network.  With  that  design  in  mind,  the 
project  was  divided  into  three  different  areas  of 
contractor  effort:  networking  technology,  simula¬ 
tor  design  and  manufacture,  and  computer  gener¬ 
ated  graphics  for  the  simulators. 

The  wide  area  network  (WAN)  that  DARPA 
used  for  SIMNET  was  based  on  a  network  that  had 
already  been  developed  for  them  by  BBN  called 
the  Terrestrial  Wide  Band  Network  (TWBnet). 
DARPA  expanded  it  into  a  network,  called  the  De- 


However,  in  1978  the  Naval  Training  Systems  Center  had  converted  its  mainframe-based  simuiation  processing  to  a  multi-minicomputer 
architecture,  a  precursor  to  distributed  processing  (2). 

J.  Thorpe,  1987,  cited  in  (2). 

In  recentyears,  as  the  number  of  entities  in  large  simulations  has  increased,  this  sort  of  broadcasting  has  become  a  bottleneck  and  has  been 
replaced  by  a  technique  called  multicasting.  See  the  section  Scalability  And  Scaling,  below. 
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fense  Simulation  Internet  (DSI),  that  could  be 
used  by  the  Defense  Department  for  simulation. 
(It  is  also  used  for  video  teleconferencing.)  Devel¬ 
opment  of  the  DSI  is  now  managed  by  the  Ad¬ 
vanced  Information  Technology  Services  Joint  Pro¬ 
gram  Office  (AITS-JPO),  a  partnership  between 
ARPA  and  the  Defense  Information  Systems 
Agency  (DISA)  established  in  1994.  ARPA  con¬ 
tinues  to  develop  the  technology;  and  DISA  will 
have  responsibility  for  operating  the  DSI  after  it 
becomes  operational.  In  February  1995,  the  DSI 
Program  Manager  (PM),  Stuart  Milner  of  ARPA, 
handed  over  the  program  management  responsi¬ 
bility  to  Commander  Walt  Corliss  of  DISA,  rec¬ 
ognizing  that  the  network  communications 
infrastructure  had  become  more  stable  and  reli¬ 
able.  The  concentration  of  the  DSI  PM  has  shifted 
to  processes,  procedures,  and  exercise  support. 
ARPA’s  Real-time  Information  and  Technology 
Networking  (RITN)  research  and  development 
program  will  continue  to  feed  new  technology 
into  the  DSI  (69). 

The  DSI’s  lines  are  being  upgraded  for  better 
communications  and  continually  extended  to  in¬ 
clude  more  “nodes,”  or  locations  where  one  or 
more  simulators  networked  together  on  a  local 
area  network  (LAN).  A  LAN  connects  the  simula¬ 
tors  at  a  particular  site  to  one  another  and  to  the 
WAN  (the  DSI).  In  late  1994  the  DSI  had  about 
100  nodes.  Once  operational,  the  DSI  will  become 
part  of  the  Defense  Information  Systems  Network 
(DISNet),  which  will  “migrate”  to  become  part  of 
the  Defense  Information  Infrastructure  (DU)  and 
will  be  interfaced  with  the  Global  Command  and 
Control  System  (GCCS),  so  that  commanders  can 
train  “at  home”  using  their  wartime  command 
and  control  systems.  DoD’s  draft  Modeling  and 
Simulation  Master  Plan  calls  for  greater  exploita¬ 
tion  of  wireless  technology,  including  satellite 
communications,  so  that  forces  may  participate  in 
simulation  even  when  deployed  overseas,  and  to 
link  live  forces  into  distributed  simulations,  as 
was  demonstrated  in  the  1994  STOW-E  exercise 
(see  box  5). 

The  decision  to  use  distributed  data  processing 
and  the  network  concept  required  a  new  architec- 
mre  for  data  communication  over  the  network  that 


would  allow  information  to  be  passed  back  and 
forth  between  hundreds  of  simulators  fast  enough 
to  convey  all  the  necessary  interaction  in  real  time. 
The  DIS  architecture  that  evolved  is,  even  more 
than  the  computer-generated  visuals  or  the  selec¬ 
tive  fidelity  simulators,  the  great  legacy  of  the 
SIMNET  program. 

The  DIS  architecture  is  an  open  architecture 
based  on  the  International  Standards  Organiza¬ 
tion’s  Basic  Reference  Model  for  Open  Systems 
Interconnection  (OSI):  ISO  7498-1984.  The  ISO 
OSI  networking  standard  provides  a  seven-layer 
structure  that  specifies  standards  and  protocols 
covering  everything  from  the  physical  network  to 
the  data  interchange  between  simulators.  There 
are  specific  standards  for  communication  over  the 
long-distance  network,  for  electronic  commu¬ 
nication  for  each  particular  type  of  local  area  net¬ 
work,  and  for  the  particular  application,  in  this 
case  the  SIMNET  simulation.  In  terms  of  this 
seven-layer  standard,  the  SIMNET  protocols  are 
the  application  layer  (the  top  layer,  layer  seven) 
that  uses  the  services  provided  by  the  network  and 
data  link  layers  without  necessarily  requiring  ser¬ 
vices  from  the  intennediate  layers  (113).  The  stan¬ 
dard  provides  common  rules  by  which 
independent  users  can  play  on  the  net. 

An  important  recent  development  is  the  estab¬ 
lishment,  by  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology,  of  the  DMSO  Archi¬ 
tecture  Management  Group  (AMG)  to  oversee  de¬ 
velopment  of  the  High-Level  Architecmre  (HLA) 
called  for  by  the  DoD  Modeling  and  Simulation 
Master  Plan  draft  of  January  1995.  The  AMG’s 
membership  was  approved  by  the  DoD  Executive 
Council  on  Modeling  and  Simulation  (EXCIMS) 
on  Mar.  22,1995,  and  the  AMG  held  its  first  meet¬ 
ing  on  Mar.  31, 1995.  Over  the  course  of  the  next 
year,  the  AMG  will  review  and  evaluate  a  series  of 
prototypes  conducted  to  test  and  further  define  the 
architecture.  Based  on  the  results  of  these  proto¬ 
types  and  other  analyses  as  necessary,  the  AMG 
will  prepare  a  baseline  definition  of  the  HLA, 
which  is  due  in  June  1996.  This  architecture  will 
be  recommended  to  the  EXCIMS,  who,  after  ap¬ 
propriate  review,  will  submit  it  to  the  Under  Sec¬ 
retary  of  Defense  (Acquisition  and  Technology) 
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BOX  5:  Synthetic  Theater  ot  War 


Synthetic  Theater  of  War  (STOW)  is  a  program  managed  by  the  Advanced  Research  Projects 
Agency  (ARPA).  It  aims  to  demonstrate  a  capability  for  large-scale  (ultimately  theater-level)  battlefield 
simulation  using  advanced  distributed  simulation  (ADS)  to  combine  virtual,  constructive,  and  live  simu¬ 
lated  combat  onto  one,  interactive  electronic  battlefield.  The  program  has  several  objectives:  1)  im¬ 
prove  the  quality  of  simulations  through  entity  level  resolution  of  combat  and  environmental  representa¬ 
tions  in  simulation,  2)  Improve  simulation  training  effectiveness  and  flexible  interfaces  with  operational 
command,  control,  communications,  computers,  and  intelligence  systems,  3)  integrate  distributed  live, 
virtual  and  constructive  simulation  models,  4)  reduce  the  overhead  costs  of  simulation  knowledge 
based,  semiautomated  forces,  5)  provide  faster  databases  with  improved  information  transfer,  6)  im¬ 
prove  after  action  and  analysis  tools,  and  7)  provide  improved  simulation-driven  crises  rehearsal  capa¬ 
bilities  to  warfighters.' 

STOW  was  originally  recommended  by  General  Paul  F.  German  and  the  Defense  Science  Board 
in  1992  as  one  of  12  experiments  to  prove  the  usefulness  of  advanced  DIS  technology  to  a  variety  of 
DoD  organizations  responsible  for  weapon  system  acquisition  and  joint  warfighting.  The  project  was  at 
first  called  the  Southwest  Experiment,  because  all  the  military  services  had  instrumented  ranges  in  the 
southwestern  United  States  that  could  be  linked  together  to  create  a  joint  (i.  e.,  multiservice)  synthetic 
battlefield.^The  objective  was  to  link  several  instrumented  training  and  test  ranges  with  simulator  sites 
to  test  proposed  weapons  and  joint  warfare  doctrine  using  live,  constructive,  and  virtual  simulation. 

In  August  1992,  Director  of  Defense  Research  and  Engineering  Victor  Reis  signed  a  Memoran¬ 
dum  of  Agreement  with  the  Joint  Chiefs  of  Staff  (JCS)  to  “formulate  and  pursue”  the  recommended 
technology  demonstrations.  The  agreement  assured  the  emphasis  on  joint  war  fighting. 

Operators  reviewed  the  original  experiments,  and  eventually  six  categories  of  demonstrations 
were  funded:  the  synthetic  battlefield,  force  generation,  F-1  6/Joint  Electronic  Combat  Electronic  Warfare 
Simulation,  commercial  battle  games,  Special  Operational  Forces/Joint  Task  Force  integration,  and  the 
Warbreaker  program,  which  included  demonstrations  of  technology  for  Theater  Area  Missile  Defense 
and  for  finding  and  attacking  Critical  Mobile  Targets.  The  JCS  allocated  $11.5  million  through  1995  for  a 
synthetic  battlefield  demonstration  that  would  involve  all  the  Commanders  in  Chief  (CINCs)  of  combat¬ 
ant  commands  and  all  the  military  services. 

The  Joint  Staff  Operational  Plans  and  interoperability  Directorate  (J-7)  joined  with  ARPA  to  use  the 
demonstrations  to  “find  practical  ways  in  which  ADS  can  improve  joint  doctrine,  plans,  operations,  train¬ 
ing,  and  education.  ”'The  alliance  combines  the  resources  and  operational  requirements  pull  of  the 
CINCs  with  development  of  a  technology  that  promises  to  help  satisfy  the  new  initiatives  in  joint  war 
fighting  exercises  mandated  by  Congress  with  the  new  East  Coast  simulation  center  and  the  new  mis¬ 
sion  training  responsibilities  of  the  U.S.  Atlantic  Command  (USACOM)  specified  in  the  Unified  Com¬ 
mand  Plan, 


’Department  of  Defense,  Advanced  Research  Projects  Agency,  Advanced  Systems  Technology  Office,  “Synthetic  Theater 
Of  War,  ”  as  of  May  12, 1995,  <URL:http://www.aipa.mil/asto/stow.html>. 

“Preference  for  center  location  later  shifted  toward  South  Carolina.  USACOM,  Influenced  by  the  Navy  and  Marine  Corps, 
preferred  South  Carolina,  where  the  Marine  Corps  has  a  warfare  center  and  the  Navy’s  Battle  Force  Tactical  Trainer  and  naval  bases 
are  located. 

*Maj,  Gen,  A.  Rogers,  Joint  Staff,  J-7,  presentation.  National  Training  Systems  Association  conference,  Alexandria,  VA,  March 
25, 1993. 


Distributed  Interactive  Simulation  of  Combat  29 


BOX  5;  Synthetic  Theater  of  War  (Cont’d.) 


Three  major  technology  demonstrations  have  been  scheduled  for  STOW,  each  a  milestone  mark¬ 
ing  the  end  of  a  phase  of  the  ADS  current  technology  demonstration  plan.  The  first,  Synthetic  Theater  of 
War-Europe  (STOW-E),  was  conducted  from  November  4  through  November  8,  1 994,  during,  and  as 
part  of,  the  annual  U.S.  Army  Forces  Europe  JTF  exercise  Atlantic  Resolve  ’94  (the  successor  to  RE¬ 
FORGER).  It  demonstrated  the  linking  and  interaction  of  live,  virtual,  and  constructive  simulations,  and 
marked  the  end  of  phase  I  of  the  ADS  current  technology  demonstration  plan.  About  3,500  entities— air¬ 
craft,  ships,  armored  vehicles,  etc.— participated.  The  maximum  number  of  entities  participating  simul¬ 
taneously— the  “high-water  mark”— was  over  2,000.  The  first  technical  papers  reviewing  achievements 
and  problems  are  beginning  to  appear  as  this  background  paper  goes  to  press.' 

STOW-E  stemmed  from  a  request  by  General  Maddox,  Commander  in  Chief  of  U.S.  Army  Forces 
Europe,  who,  having  SIMNET  simulators  and  the  live  range  of  the  Combat  Maneuver  Training  Center 
(CMTC)  In  Germany  available,  wanted  to  be  able  to  use  the  capabilities  of  simulated  training  to  tie  to¬ 
gether  a  SIMNET  battalion,  constructive  forces,  and  live  forces  on  the  instrumented  range  to  do  large- 
scale  exercises.  In  Germany  today  for  political,  economic,  and  ecological  reasons  it  is  not  possible  to 
maneuver  live  units  larger  than  a  battalion. 

In  STOW-E,  constructive  simulations,  such  as  the  Brigade/Battalion  Battle  Simulation  (BBS),  were 
linked  to  the  simulation  network  via  a  translation  unit  (called  an  Advanced  Interface  Unit  or  Advanced 
Translator  System),  which  translated  constructive  simulation  results  into  DIS-compatible  messages.  The 
translators  were  designed  to  disaggregate  a  platoon  (for  example)  that  is  simulated  as  an  aggregate 
unit  by  the  BBS  into  four  tanks  visible  to  SIMNET  simulators  whenever  a  SIMNET  simulator  approached 
the  agregated  platoon  close  enough  to  see  one  of  Its  tanks.  STOW-E  also  used  ModSAF,  a  DlS-compat- 
Ible  constructive  simulation.  Translators  were  also  used  by  the  participating  field  instrumentation  sys¬ 
tems,  such  as  the  CMTC  Instrumentation  System,  to  report  the  locations  and  status  of  real  vehicles  to 
the  simulation  network  so  that  representations  of  them  could  be  seen  by  simulator  crews.  Several  band¬ 
width  reduction  techniques-designed  to  reduce  network  congestion  and  permit  more  entitles  to  be 
simulated— were  demonstrated  in  STOW-E. 

The  next  major  technology  demonstration  planned  for  STOW  is  Synthetic  Theater  of  War-1997 
(STOW-97),  to  be  sponsored  by  USACOM,  focused  at  the  Joint  Task  Force  level,  and  conducted  in 
1997,  In  late  1994,  the  projected  scale  of  STOW-97  was  revised  downward  from  100,000  entities  to 
50,000  entities.  The  third  and  final  major  demonstration  planned  for  STOW  is  Synthetic  Theater  of 
War-2000  (STOW-2000),  to  be  sponsored  by  USACOM  and  conducted  in  2000.  It  will  mark  the  end  of 
phase  III  of  the  technology  demonstration  plan,  i.e.,  of  technology  insertion  and  transition  to  operational 
systems. 


^University  of  Centra!  Florida,  Institute  for  Simulation  and  Training, f2f/7  DIS  Standards  Workshop  Proceedings,  July  14,  1995, 
available  online  at  <URL:ftp://ftp.sc.ist.ucf.edu/public/STDS/workshop/12th/proceedings/>  .  See  also  P.  Marion,  “STOW"E*First  Im¬ 
pressions  Report,”  Nov.  28, 1994,  available  online  at  <URL:gopher://triton.dmso.mil.:70/00/twg/dsi/dsi-docs/stow-e.txt> 

SOURCE:  Office  of  Technology  Assessment,  1995. 


for  approval.  During  the  course  of  its  work,  the 
AMG  will  brief  DoD’s  Modeling  and  Simulation 
Working  Group,  which  supports  the  EXCIMS,  on 
its  activities  (165,166,1  12,180). 


I  Uses  and  Limitations  of  SIMNET 

SIMNET  matured  as  a  company-level  simulation 
and  now  easily  handles  battalion-level  actions. 
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The  primary  controversy  surrounding  SIMNET  is 
the  degree  to  which  it  is  possible,  or  even  desir¬ 
able,  to  create  a  large  battle  by  connecting  SIM¬ 
NET  nodes  (and  their  other-service  equivalents, 
as  these  develop)  together  (see  e.g.,  box  6).  While 
intuitively  one  might  think  that  the  whole  is  the 
sum  of  the  parts  and  that  therefore  with  enough 
SIMNET  simulators  one  could  simulate  a  large 
battle,  other  considerations  may  supervene. 

For  example,  SIMNET  focused  on  tanks  and 
other  vehicles,  and  DIS  does  not  yet  handle  the  in¬ 
dividual  soldier  well.  Yet  the  humble  foot  soldier, 
modemly  equipped,  retains  his  key  role  in  war¬ 
fare.  The  considerable  antitank  potential  of 
today’s  infantry  makes  the  foot  soldier  an  impor¬ 
tant  player  on  the  tanks’  battlefield  and  the  key  to 
the  tanks’  survival  should  they  venture  into  woods 
or  urban  areas.  The  Army’s  Integrated  Unit  Simu¬ 
lation  System  (lUSS)  project  and  Individual  Port 
(I-Port)  project  have  demonstrated  that  individual 
dismounted  infantry  troops  can  participate  in  syn¬ 
thetic  combat  (147),  and  the  Army  recently 
awarded  a  $1  million  contract  to  Avatar  Partners, 
Inc.,^^  of  Boulder  Creek,  California,  for  the  devel¬ 
opment  of  a  wireless  unencumbered  virtual  reality 
system— called  Dismounted  Infantry  Virtual  En¬ 
vironment  (DIVE) — for  infantry  training  applica¬ 
tions.  (121)  However,  it  may  not  be  feasible, 
affordable,  or  necessary  (for  some  training  tasks) 
to  populate  a  synthetic  battlefield  with  as  many 
dismounted  troops  as  there  would  be  on  a  real 
battlefield. 


Another  problem  is  that  most  participants  in  a 
battle  spend  much  more  time  waiting  than  fight¬ 
ing.  Moreover,  battle  is  a  stochastic  (i.e.,  random) 
process — actually,  the  result  of  many  stochastic 
processes.  To  deduce  valid  results  from  a  simula¬ 
tion,  it  would  be  necessary  to  simulate  the  sto¬ 
chastic  processes^^  and  to  do  so  many  times,  so 
that  typical  outcomes  and  the  range  of  outcomes 
can  be  discemed.^^  Considering  the  hours  of  inac¬ 
tivity  that  most  trainees  would  endure  in  an  accu¬ 
rate  simulation,  it  is  questionable  whether  the 
troops’  training  time  (and  simulator  time)  would 
be  well  spent  in  repeating  the  same  scenario  many 
times  in  order  to  estimate  the  expected  outcome 
(e.g.,  losses)  and  the  range  of  outcomes.  Perhaps, 
if  the  goal  were  a  simulation  of  a  large  battle,  all  of 
the  forces  should  be  semiautomated,  not  just  one 
side’s.  Indeed,  computerized  combat  models  fea¬ 
turing  large  forces,  in  some  cases  resolved  down 
to  the  vehicle  level,  have  been  available  for  some 
time.  These  include  Corps  Battle  Simulation, 
Concepts  Evaluation  Model,  Janus,  Joint  Inte¬ 
grated  Contingency  Model,  TACWAR,  etc.^'* 
SIMNET’s  graphical  interface  will  be  wasted  on  a 
higher  echelon  commander,  whose  personal  point 
of  view  will  be  the  inside  of  a  command  post  re¬ 
gardless  of  whether  the  battle  is  fought  inside  a 
computer,  in  SIMNET,  at  NTC,  or  in  the  real 
world. 

SIMNET’s  use  for  large  battles  has  been  pro¬ 
posed  for  two  different  purposes.  One  is  exercise. 


Avatar  Partners,  Inc,,  is  now  called  DIVE  Laboratories,  Inc.  DIVE  Laboratories,  Inc.,  will  al.so  assist  General  Reality  Company  in  an 
ettojl  to  “develop  a  VR-based  dispersed  command  post  architecture  for  military  commanders,”  for  which  General  Reality  Company  was 
awarded  a  Phase  1  Small  Business  Innovation  and  Research  (SBIR)  contract  from  US  Army  CECOM  (47) . 

^^In  SIMNET,  this  is  done  by  letting  hmnans  control  some  processes,  such  as  aiming  and  tiring  weapons.  Otlier  stochastic  processes,  such 
as  wind  velocity,  are  simulated  by  computers  using  Afrjnte  Carlo  methods — in  effect,  throwing  dice  electronically.  The  result  of  a  roll  of  a  pair  of 
electronic  dice  is  adjusted  to  give  the  right  average  speed  and  gustiness — i.e.,  range  or  standard  deviation  of  speed.  For  example,  the  average 
value  rolled  by  a  pair  of  dice  is  7 ;  values  range  from  2  to  12,  with  a  standard  deviation  of  about  1 .7 1 .  In  order  to  model  a  westerly  wind  witli  an 
average  speed  of  14  knots,  gusting  to  17.5  knots,  one  might  double  the  value  rolled  to  get  the  speed  in  knots. 

The  result  of  a  single  simulation  might  be  valid  in  the  sense  that  it  could  occur  in  reality,  but  it  might  be  so  unrepresentative  of  outcomes  of 
similar  conflicts  that  one  would  be  foolish  to  assume  tliat  it  is  a  reliable  prediction  of  what  will  happen,  without  some  knowledge  of  the  range  of 
outcomes  that  occur  in  several  runs  of  the  simulation  and  without  some  evidence  tliat  outcomes  of  comparable  real-world  situations  usually  lie 
within  the  range  of  simulation  outcomes. 

Some  of  these  models  were  developed  at  Federally  Funded  Research  and  Development  Centers  (FFRDCs),  fonnerly  called  Federally 
Chartered  Research  Corporations  (FCRCs). 
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BOX  6:  DFITII 


At  the  time  SIMNET  was  being  developed,  the  Navy  was  working  to  develop  a  naval  gunnery 
training  system,  also,  called  Battle  Force  In-port  Training  (BFIT),  later  renamed  Battle  Force  Tactical 
Training  (BFTT).  The  Navy  was  working  on  networking  ships  in  port.  The  Defense  Advanced  Research 
Projects  Agency  (DARPA)  combined  SIMNET  networking  technology  and  simulators  with  BFIT  to  dem¬ 
onstrate  component  interoperability  (a  first  step  towards  joint  warfighting  training)  the  following  month  in 
April  1990. 

The  BFIT  demonstration  networked  a  Navy  ship,  the  U.S.S.  Wasp,  in  port  at  Norfolk,  Virginia,  with 
Marine  Corps  SIMNET  simulated  helicopters  at  Fort  Rucker,  Alabama,  simulated  Army  tanks,  Navy  staff 
command  centers,  and  an  observation  node  at  the  Institute  for  Defense  Analyses.  The  demonstration 
included  a  Marine  Corps  pilot  and  Army  tank  commander  calling  for  Navy  fire  of  a  Tomahawk  missile 
over  simulated  terrain  (using  the  Fort  Hunter-Liggett,  California,  terrain  database).  The  simulated  Toma¬ 
hawk  mission  required  command,  control,  and  communication  between  Navy,  Army,  and  Marine  Corps 
elements  to  coordinate  missile  flight  through  friendly  forces  to  its  target.  The  simulated  Tomahawk  mis¬ 
sile  showed  up  on  the  Navy  radar  as  did  the  pilots  when  they  flew  over  the  U.S.S,  Wasp  symbol  on  their 
screens.  It  was  the  first  time  the  Marine  Corps,  Navy,  and  Army  played  on  the  same  network  using  their 
own  systems.  'The  joint  operation  was  not  completely  smooth,  but  improved  with  practice. 

The  demonstration  not  only  illustrated  the  DIS  technology’s  joint  component  interoperability,  but 
also  illustrated  some  technical  accomplishments  and  features  of  the  technology.  Two  valuable  features 
of  DIS  technology  are  its  modularity  and  its  flexibility.  Ultimately,  any  type  of  computer  system  that  can 
do  the  processing  and  display  can  be  brought  onto  the  network  and  added  to  the  interaction  (within  the 
limits  of  the  number  of  sites  supported  by  the  networking  technology).  In  the  DARPA/Navy  BFIT  proof- 
of-principle  demonstration  the  computer  consoles  were  real  consoles  in  real  ships,  turning  real  equip¬ 
ment  into  simulators.  This  type  of  simulation,  with  outputs  from  simulation  interactions  used  as  inputs  for 
the  visual  displays  of  real  equipment  such  as  sonar  and  radar  screens,  is  called  stimulation.  So  not  only 
were  geographically  dispersed  simulators  networked  in  real  time,  but  simulators  designed  for  interactiv¬ 
ity  were  also  networked  with  real  equipment  not  specifically  designed  for  interactive  training.  This  was 
demonstrated  on  a  larger  scale  in  the  1994  STOW-E  exercise.  A  milestone  yet  to  come  is  the  routine 
incorporation  of  equipment  for  interacting  in  simulation  (called  field  instrumentation  in  the  DIS  communi¬ 
ty)  into  platforms  such  as  tanks  and  airplanes. 


'BFIT II  Battle  Force  In-Port  Training  1992  (videocassette),  VO  92030,  U.S.  Navy,  Naval  Ocean  Systems  Center,  CA,  1992. 
SOURCE:  Office  of  Technology  Assessment,  1995. 


the  other  is  simulation.  At  present,  the  large  com¬ 
puterized  combat  models  are  used  in  these  ways. 
In  command  post  exercises  CPXs  they  deter¬ 
mine  the  combat  outcomes  resulting  from  the 
commanders’  decisions.  As  simulations,  they  are 
used  to  develop  answers  to  questions  such  as  force 
sizing  and  deployment.  In  the  latter  vein,  one  pos¬ 
sible  use  of  SIMNET  is  “virtual  prototyping,  ”  in 
which  a  candidate  weapon  would  take  its  place  on 
the  simulated  battlefield  before  it  is  even  pro¬ 


duced,  so  that  its  utility  could  be  evaluated  prior  to 
the  effort  and  expense  of  production. 

SIMNET  proponents  argue  that  the  older  com¬ 
bat  models  lack  the  humans  necessary  to  simulate 
human  behavior.  SIMNET  is  constructed  to  pres¬ 
ent  its  trainees  with  a  realistically  threatening  en¬ 
vironment,  using  sights  and  sounds  to  raise  the 
tension  level  as  much  as  conveniently  possible. 
The  sight-and-soimd  presentation  created  by 
SIMNET  is  counted  by  many  as  “virtual  reality” 
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or  a  “synthetic  environment.”  Therefore,  it  is  ar¬ 
gued,  SIMNET  forces  will  display  behavior  more 
human  than  the  behavior  programmed  into  the 
forces  that  inhabit  pure  computer  models. 

This  point  begs  the  question  of  whether  the 
SIMNET  players,  regardless  of  how  tense, 
aggressive,  or  scared  SIMNET  can  make  them, 
will  really  behave  as  they  would  if  they  were  in 
mortal  danger.  A  related  question,  which  is  impor¬ 
tant  in  simulations  conducted  for  research  (rather 
than  training),  is  whether  the  participants  are  as 
qualified,  trained,  and  experienced,  as  will  be 
those  who  would  fight  a  real  war.  This  issue  is  not 
peculiar  to  DIS;  it  has  been  raised  earlier  in  the 
context  of  CPXs,  other  constructive  simulations, 
FLEETEXs,  and  crisis  simulation  games,  many  of 
which  (for  various  reasons,  including  training)  are 
played  by  participants  much  less  experienced  or 
distinguished  than  those  they  attempt  to  emulate. 

The  argument  that  human  participation  is  nec¬ 
essary  for  realism  seems  difficult  to  reconcile  with 
acceptance  (by  proponents  of  that  argument)  of 
the  use  of  CGF.  If  the  CGF  don’t  behave  realisti¬ 
cally,  the  battle  is  only  half-realistic  in  this  regard, 
and  if  they  do  then  CGF  could  be  used  as  friendly 
forces,  in  simulations  conducted  for  research  or 
the  provide  realism  and  context  in  simulations 
conducted  for  training.  This  apparent  contradic¬ 
tion  is  not  necessarily  a  real  contradiction  in  simu¬ 
lations  conducted  for  training,  in  which  changing 
the  behavior  of  the  trainees  under  certain  condi¬ 
tions  is  the  objective,  or  if  semiautomated  forces 
(rather  than  CGF)  are  used. 

FIDELITY,  FOG,  and  FRICTION 

Computerized  combat  models  are  often  received 
with  skepticism — ^how  can  a  computer  program 
possibly  capture  such  a  complex  activity  as  com¬ 
bat?  The  quality  of  SIMNET’s  synthetic  environ¬ 
ment,  like  that  of  a  stereo,  is  judged  in  terms  of 
fidelity.  Music  lovers  sometimes  have  an  opportu¬ 
nity  to  compare  the  output  of  their  stereos  to  the 


real  thing  (as  they  recall  it)  just  by  listening,  and 
a  stereo  manufacturer  or  broadcast  or  recording 
engineer  can  assess  fidelity  by  measuring  the 
sound  content  of  real  music  and  of  his  product’s 
output.  SIMNET  designers  were  under  no  illusion 
that  they  could  fool  trainees  into  actually  believ¬ 
ing  they  were  in  tanks  in  battle,  and  they  recog¬ 
nized  the  need  to  put  their  efforts  where  they 
would  do  the  most  good.  Thus  they  decided  to 
measure  their  success  in  terms  of  the  degree  to 
which  the  trainees’  behavior  resembled  that  of  ac¬ 
tual  tank  crews,  and  to  adopt  a  selective  fidelity  ap¬ 
proach  in  which  only  the  details  that  proved  to  be 
important  in  shaping  behavior  would  be  repli¬ 
cated.  Cost  considerations  received  their  due,  and 
thus  the  inside  of  the  SIMNET  tank  simulator 
could  well  be  termed  a  nunimalist  representation 
of  a  tank’s  interior,  while  the  sounds  presented  by 
SIMNET  are  quite  realistic:  sound  effects  are 
cheap,  and  people  react  strongly  to  them.  Espe¬ 
cially  evocative  of  reality  aie  the  low-frequency 
“sounds”  delivered  directly  to  the  seat  of  the  driv¬ 
er’s  pants:  these  verge  on  being  tactile  stimuli 
rather  than  sounds,  and  convincingly  recreate  the 
sense  of  driving  over  uneven  terrain. 

Users  of  SIMNET  and  other  virtual  reality  sys¬ 
tems  almost  universally  report  an  initial  dismay  at 
the  apparent  lack  of  fidelity,  followed  by  accep¬ 
tance  and  a  report  that  “it  seemed  so  real”  when 
they  start  to  interact  with  the  program  and  get  a 
sense  of  involvement.^^  This  feeling  can  be  repli¬ 
cated  at  any  video  game  arcade. 

One  can  easily  carp  at  particular  gaps  in  real¬ 
ism,  and  any  such  observations  should  be  preced¬ 
ed  by  consideration  of  the  selective  fidelity 
concept  and  the  definition  of  fidelity  as  percentage 
of  behavior  learned.  For  example,  the  German  use 
of  cars  dressed  up  as  tanks  in  their  prewar  maneu¬ 
vers  was  a  shortcoming  in  realism  but  did  not  stop 
the  German  tank  crews  from  deriving  consider¬ 
able  training  benefit  and  the  German  commanders 


SIMNET  workers  feel  that  the  ultimate  accolade,  sometimes  given  after  a  live  field  exercise,  is  a  soldier’s  statement  that  “That  felt  just 
like  in  the  simulator!’’ 
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from  honing  their  Blitzkrieg  theory  of  warfare 
(17). 

Yet  the  gaps  in  fidelity  are  there.  Even  the  Na¬ 
tional  Training  Center,  which  affords  the  trainees 
real  tanks,  real  terrain,  real  weather,  and  human 
opponents,  has  been  criticized  on  the  basis  of  re¬ 
sidual  artificialities  such  as  the  inability  of  its  laser 
“bullets”  to  penetrate  smoke  and  dust.  Such  criti¬ 
cisms  of  NTC,  SIMNET,  or  other  high-tech  simu¬ 
lations  often  evoke  the  response  that  the  low-tech 
alternatives,  such  as  shooting  at  white  paper  tar¬ 
gets,  are  even  less  realistic.  Some  skeptics  reject 
even  this  argument,  maintaining  in  effect  that  “a 
little  fidelity  is  a  dangerous  thing”  and  that  the 
convincing  realism  of  high-tech  simulation  could 
or  does  perniciously  foster  negative  learning  in 
which  the  trainees  adapt  to  the  simulation,  artifi¬ 
cialities  and  all,  to  the  detriment  of  actual  field 
performance  (see  box  7). 

Accuracy  and  fidelity  must  be  considered  in 
context.  SIMNET’s  sketchy  presentation  of  the 
interior  of  the  tank  is  accepted  because  the  pur¬ 
pose  of  SIMNET  is  not  to  familiarize  tank  crews 
with  the  interiors  of  their  tanks,  but  to  give  them 
experience  in  using  a  tank  that  they  already  know 
how  to  drive.  Indeed,  for  some  purposes  one  could 
validly  dispense  with  all  the  high-tech  equipment 
and  revert  to  model  tanks  on  a  sand  table.  An 
American  unit  commander  in  Desert  Storm  did 
just  that  to  coordinate  the  movement  of  his  ve¬ 
hicles  and  avoid  a  traffic  jam. 

Use  of  SIMNET  to  create  large  simulated 
battles  could  decrease  fidelity  by  departing  from 
what  SIMNET  does  best — intense  armored  com¬ 
bat  at  the  company  or  battalion  level.  The  larger 
battlefield  requires  that  the  soldier  perform  many 
other  tasks,  such  as  repair  and  maintenance  of  his 
equipment  and  of  himself  (including  sleeping), 
ensuring  the  security  of  rear  areas,  and  just  plain 
living  outdoors.  SIMNET  does  not  allow  for  the 
simulation  of  these  tasks. 


If  the  goal  is  to  provide  higher  echelon  officers 
with  training  and  exercise  of  their  command  func¬ 
tions,  use  of  SIMNET  is  not  necessary:  a  CPX, 
with  combat  resolution  provided  by  some  other 
means,  would  afford  a  realistic  training  environ¬ 
ment  for  the  officers.  If,  on  the  other  hand,  the  goal 
is  to  fight  large  battles  in  SIMNET  to  find  out  how 
large  battles  work  (a  laudable  goal — for  all  the  ef¬ 
fort  expended  by  humanity  on  war,  land  battles  re¬ 
main  poorly  understood)  or  to  experiment  with 
new  formations,  equipment,  or  force  structure, 
then  SIMNET  or  something  of  its  ilk  may  be  the 
right  tool,  if  one  can  be  sure  that  results  are  accu¬ 
rate. 

Clausewitz  cited  the  great  difficulty  of  obtain¬ 
ing  correct  information  on  the  batdefield  and 
dubbed  it  “the  fog  of  war”  (16).^^  Traditional 
combat  models  have  often  been  faulted  for  being 
idealized  to  the  point  of  omitting  the  fog  of  war. 
SIMNET  and  other  DIS,  through  their  use  of  mul¬ 
tiple  people  and  portrayal  of  some  communica¬ 
tions  processes,  promise  to  replicate  the  fog  of  war 
better  than  have  previous  models.  In  one  SIMNET 
run  witnessed  by  OTA  observers,  three  Ml 
Abrams  tanks  covered  the  retreat  of  four  M-2 
Bradley  fighting  vehicles  that  had  been  deployed 
far  forward  as  scouts.  The  four  Bradleys  safely 
reached  the  main  U.S.  position,  a  ridge  line  occu¬ 
pied  by  hull-down  friendly  vehicles.  The  return¬ 
ing  Mis  were  not  so  lucky:  one  was  destroyed  by 
enemy  fire,  one  was  mistaken  for  an  advancing  en¬ 
emy  and  became  an  instance  of  fratricide,  and  the 
third  became  disoriented  and  lined  up  on  the 
wrong  side  of  the  ridge.  Ordered  to  proceed  to  the 
rear,  it  turned  around  and  proceeded  forward  and 
was  promptly  demolished  by  the  advancing  en¬ 
emy.  A  traditional  computer  model  would  have 
been  hard  pressed  to  produce  such  a  set  of  occur¬ 
rences.  However,  the  use  of  semiautomated  forces 
threatens  to  remove  the  human  element  and  rein¬ 
troduce  unrealistic  clarity. 


Clausewitz’s  classic  work,  Vom  Kriege  [On  War],  was  first  published  posthumously  by  his  wife  in  1832. 
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BOX  7:  Negative  Learning  Transfer 


Providing  and  manipulating  the  virtual  environments  of  trainees  is  done  to  promote  the  learning  of 
behavior  that  will  be  useful  on  the  battlefield  and  to  promote  the  retention  and  transfer  of  those  skills  to 
the  real  battlefield.  However,  the  effectiveness  of  the  training  Is  limited  by  the  understanding  of  the  de¬ 
signers  of  the  virtual  environment.  Error  or  inadequate  fidelity  can  cause  negative  learning  transfer:  the 
learning,  in  simulation,  of  behavior  that  is  rewarded  with  success  on  the  simulated  battlefield  but  is  in¬ 
appropriate  on  a  real  battlefield.  This  possibility  is  of  particular  concern  to  designers  and  users  of  com¬ 
bat  aircraft  mission  simulators.  Split-second  timing  is  critical  in  some  phases  of  strike  missions  and  de¬ 
fensive  counter-air  missions.  Where  the  motor  skills  coordination  must  be  precise,  the  simulator  must  be 
equally  precise.  ' 

SIMNET  designers  encountered  what  they  considered  a  negative  learning  transfer  when  they  stu¬ 
died  a  SIMNET  predecessor,  the  Unit  Conduct  of  Fire  Trainer.  One  trainee  thought  he  was  greatly  im¬ 
proving  his  gunnery  until  his  trainer  kept  quiet.  Without  the  external  help  directing  his  attention  to  tar¬ 
gets,  the  gunner’s  success  rate  returned  to  his  initial  low  percentage.Mn  SIMNET,  concern  has  been 
expressed  about  negative  learning  transfer  because  the  hatches  In  SIMNET  tank  simulators  are  “but¬ 
toned  up,  ”  After  bumping  their  heads  a  couple  of  times,  commanders  might  learn  not  to  look  out  their 
hatches. 

Artificialities  in  the  live  simulation  conducted  at  the  National  Training  Center  may  also  cause  neg¬ 
ative  learning.  Because  weapons’  fire  is  simulated  with  lasers,  smoke  and  dust  not  only  provide  good 
visual  cover,  they  actually  make  the  participant  impervious  to  bullets  in  the  sense  that  a  hit  will  not  regis¬ 
ter  since  the  laser  beam  will  not  penetrate  the  obscurants.  A  training  result  might  be  an  overreliance  on 
dust  and  smoke  for  cover  and  overconfidence  under  such  conditions. 

How  serious  a  problem  is  negative  skill  transfer?  Opinions  vary,  but  it  is  generally  agreed  that  the 
benefits  of  simulation  training  and  the  advantages  it  provides  for  combat  readiness  far  outweigh  any 
negative  transfer. 

Negative  learning  can  even  happen  in  real  combat  situations.  During  World  War  II,  the  British 
engaged  the  Italians  in  North  Africa  and  learned  how  to  fight  the  Italians  successfully.  When  the  Ger¬ 
mans  arrived  to  reinforce  the  Italians,  the  British  had  all  the  wrong  fighting  skills.'Flghting  skills  Involve 
a  constant  learning  process,  some  of  it  corrective.  With  technology  changes  and  upgrades  to  equip¬ 
ment,  old  skills  have  to  be  unlearned  and  new  skills  applied.  Dealing  with  corrections  to  skill  sets  is  a 
fact  of  life  with  or  without  simulation.  Adaptability  in  combat  is,  itself,  an  essential  skill.  However,  the 
nature  of  simulation  in  training  is  such  that  small  design  mistakes  can  handicap  large  numbers  of 
people  and  any  potential  negative  skill  transfer  Is  something  that  requires,  therefore,  vigilant  watching 
for  and  correction  as  part  of  the  verification,  validation,  and  accreditation  process. 


’  Imprecise  coordination  of  computer-generated  imagery  with  simulator  base  motion  (if  any)  can  also  cause  simulator  sick¬ 
ness,  which  is  closely  related  to  motion  sickness  (seasickness,  airsickness,  etc.), 

“J.  McDonough,  interview  with  L.  Voss  at  Illusion  Engineering  Inc.,  Westlake  Village,  CA,  August  1992. 

®M.  Carver,  Dilemmas  of  the  Desert  War  (Bloomington,  IN:  University  of  Indiana  Press,  1986),  p.  16. 

SOURCE:  Office  of  Technology  Assessment,  1995. 
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Clausewitz  separately  identified  “friction”  as  a 
major  force  on  the  battlefield.  SIMNET  can  repli¬ 
cate  some  sources  of  friction,  such  as  vehicle 
breakdowns,  but  many  believe  the  real  world  of 
the  battlefield  to  harbor  a  myriad  of  misadven¬ 
tures  not  provided  for  in  SIMNET  or  other  mod¬ 
els. 

VERIFICATION,  VALIDATION,  AND 
ACCREDITATION 

Predictions  generated  by  computerized  combat 
models  are  often  received  with  skepticism — ^how 
can  a  computer  program  possibly  capture  such  a 
complex  activity  as  combat?  The  issue  here  is  the 
validity  of  models  and  simulations  and,  more  fun¬ 
damentally,  the  process — validation — by  which 
validity  is  assessed.  The  Department  of  Defense 
defines  validation  as  “the  process  of  determining 
the  degree  to  which  a  model  or  simulation  is  an  ac¬ 
curate  representation  of  the  real  world  from  the 
perspective  of  the  intended  uses  of  the  model  or 
simulation”  (153,155,185). 

A  variety  of  techniques  have  been  proposed 
and  used  to  validate  computer  models  of  combat; 
they  include  critical  analysis  of  the  data  and  com¬ 
putational  procedures  used  in  the  models  and 
comparison  of  the  outputs  (predictions)  of  models 
and  simulations  to  actual  outcomes — of  historical 
battles  (e.g. ,  see  references  34  and  1 5)  or  exercises 
(e.g.,  see  references  41  and  141) — or  to  the  predic¬ 
tions  of  other  models  and  simulations. 

Critical  analysis  of  data  for  correctness  is  an 
element  of  data  verification,  validation,  and  certi¬ 
fication  (VV&C).  Critical  analysis  of  the  com¬ 
putational  procedures  used  in  models  is  called 
structural  validation,  which  “includes  examina¬ 
tion  of  [model  and  simulation]  assumptions  and 
review  of  the  [model  and  simulation]  architecture 
and  algorithms  in  the  context  of  the  intended  use” 
(174).  Comparison  of  model  and  simulation  out¬ 
puts  to  outcomes  of  historical  battles  is  one  type  of 
output  validation,  which  “answers  questions  on 


how  well  the  M&S  results  compare  with  the  per¬ 
ceived  real  world”  (174). 

Two  related  steps  in  the  acceptance  process  are 
often  considered  along  with  validation:  verifica¬ 
tion  and  accreditation.  DoD  defines  verification 
as  “the  process  of  determining  that  a  model  or 
simulation  implementation  accurately  represents 
the  developer’s  conceptual  description  and  speci¬ 
fications”  (185).  It  establishes  that  the  computer 
code  is  free  of  technical  “bugs”  (i.e.,  errors^^). 
DoD  defines  accreditation  as  “an  official  deter¬ 
mination  that  a  model  or  simulation  is  acceptable 
for  a  specific  purpose”  (185). 

To  date,  validation  of  SIMNET  has  rested 
largely  on  users’  assertions  that  it  “feels  right.” 
Such  declarations  are  evidence  of  face  validity — 
i.e.,  that  on  the  face  of  it  the  experience  is  not  so 
unrealistic  as  to  preclude  the  intended  use.  How¬ 
ever,  such  declarations  do  not  guarantee  that  the 
effectiveness  of  the  vehicle,  crew,  or  unit  will  be  in 
combat  what  it  was  in  simulation.  In  fact,  some 
observers  worry  that  SIMNET  may  feel  too 
right — ^that  it  may  be  “seductive”  (32). 

For  example,  a  tank  commander  must  discern 
an  enemy  tank  at  a  distance  of  a  thousand  yards  or 
more,  despite  the  camouflaged  appearance  of  the 
enemy  and  his  likely  exploitation  of  concealment 
offered  by  vegetation  or  terrain.  This  task  pushes 
human  vision  to  its  limits.  Although  target  ac¬ 
quisition  feels  the  same  in  SIMNET  as  in  real  life 
(each  requiring  great  alertness  and  a  quick  reac¬ 
tion  to  the  first  sign  of  the  barely-detectable  en¬ 
emy),  real-life  target  acquisition  entails  the 
interpretation  of  visual  details  far  too  subtle  to  be 
presented  on  a  SIMNET  screen,  creating  a  dis¬ 
crepancy  in  the  average  distance  at  which  targets 
are  acquired.  Such  a  discrepancy  can  be  discov¬ 
ered  only  through  a  systematic  validation  effort, 
such  as  one  that  was  undertaken  to  validate  target 
acquisition  models  (205).  Other  discrepancies, 
such  as  the  tank  commander’s  limited  field  of 
view  in  an  Ml  simulator,  ai'e  more  obvious  and  are 


Reference  number  (78)  describes  the  origin  of  this  use  of  the  word  “bug.” 
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This  illustration  depicts  a  virtual  battlefield  scene  such  as  might  be  displayed  in  a  Close  Combat  Tactical  Trainer  simulator-except  the  armored 
vehictes  and  aircraft  are  cut  away  to  show  cut-away  views  of  the  simulators  in  which  crews  are  controlling  the  operation  of  their  simulated  vehicles  on 
the  virtual  battlefield.  Also  depicted  is  the  cabling  that  connects  the  simulators.  The  vehicles  shown  include  Abrams  tanks  (foreground  and  center),  a 
Bradley  armored  fighting  vehicle  (left,  with  infantry  troops  dismounting),  an  Apache  helicopter  (above  left),  a  U.S.  Air  Force  A-10  Waiihog  close  air 
support  aircraft  (above  left  center  and  above  right),  dismounted  infantry  troops  (center,  beyond  Abrams  tank),  and  various  enemy  vehicles  (above 
right).  Close  Combat  Tactical  Trainer  simulators  have  not  yet  been  built. 

SOURCE:  U.S.  Army,  Combined  Arms  Tactical  Trainer  Project  Manager,  1995. 

being  addressed  in  the  Army’s  Close  Combat  Tac-  about  the  Battle  of  73  Easting  (a  tank  battle  in  the 

tical  Trainer  (CCTT)  project— the  follow-on  to  Persian  Gulf  War).  DARPA  compared  the  data  to 

SMNET-T  (see  figure  6).  the  results  of  SIMNET  simulations  to  see  how 

Various  approaches  to  validating  SIMNET  closely  the  simulated  battles  compared  to  the  real 

through  comparison  of  its  results  to  presumed  one  (15).  Similai-  comparisons  have  been  per- 

facts”  can  be  imagined.  One  validation  project  formed  with  constructive  models  such  as  CAST- 

conducted  by  DARPA  collected  detailed  data  FOREM  (50),  the  Concepts  Evaluation  Model 


“It  is  sometimes  difficult  to  identify  the  real  world  to  which  a  model’s  or  simulation’s  output  should  be  compared:  “The  basic  point  about 
‘reality’  is  the  non-existence  of  general  agreement  as  to  what  it  is.. .  .  As  in  science  generally,  the  dichotomy  of  fact’  and  ‘theory’  is  an  oversim¬ 
plification”  (140).  See  also  reference  number  (174). 
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(204),  Janus(A)  (107,  109),  the  Oak  Ridge 
Spreadsheet  Battle  Model  (56),  various  Lanches- 
trian  models  (34,52,53,54,58,108),  the  Quanti¬ 
fied  Judgement  Model  (30),  and  other  models 
(94),  applied  to  battles  too  numerous  to  list  here 
exhaustively^^  but  including  the  Battle  of  Iwo 
Jima  (34),  the  Battle  of  the  Bulge  (Ardennes  1944) 
(204),  the  Inchon-Seoul  Campaign  (52),  the  Six- 
Day  War  of  1 967  (30),  the  1 973  War  of  Atonement 
(30),  the  1982  War  of  the  Cherry  Blossoms  in  the 
Bekaa  Valley  of  Lebanon  (30),  the  Battle  of  73 
Easting  in  the  Persian  Gulf  War  (50,107),  and  the 
World  War  II  U-boat  campaign  in  the  Bay  of 
Biscay.  In  current  DoD  terminolgy,  these  compar¬ 
isons  would  be  described  as  output-validation  ac¬ 
tivities  intended  to  validate  (or  to  invalidate)  the 
models  for  predictive  uses  by  comparing  their  pre¬ 
dictions  to  historical  outcomes. 

This  approach  to  validation  is  not  quite  as  easy 
at  it  sounds.  One  problem  is  obtaining  good  data 
for  such  an  undertaking;  this  may  be  impossible 
(e.g.,  if  one  is  interested  in  validating  a  model  of 
strategic  nuclear  war  between  superpowers)  or 
could  require  more  time  and  money  than  did  creat¬ 
ing  the  model  in  the  first  place.  Those  who  attempt 
it  may  be  subjected  to  the  unenlightened  charge 
that  it  is  waste  of  money  to  predict  the  past.  Anoth¬ 
er  possible  pitfall  is  that  if  modelers  use  all  appli¬ 
cable  historical  cases  to  create  the  model  (56), 
comparing  its  predictions  to  those  outcomes  may 
show  what  looks  like  a  good  fit  to  the  layman,  but 
it  would  have  no  goodness  of  fit  in  a  statistical 
(i.e.,  scientific)  sense,  and  the  model’s  predictions 
would  be  of  uncertain  accuracy  and  reliability.  A 
statement  such  as  “we  have  calibrated  our  model 
to  the  results  of  the  1973  Arab-Israeli  war,  and  it 
can  reproduce  them  perfectly”  (62,  p.  145)  im¬ 
plies  that  the  model  has  descriptive  validity — i.e., 
it  explains  the  outcome  of  the  combat  (22) — ^but 
the  statement  provides  no  confidence  in  the  reli¬ 
ability  with  which  the  model  will  predict  the  out¬ 
come  of  a  similar  future  war,  and  other  models 


may  be  able  to  be  calibrated  to  explain  the  out¬ 
come  equally  well. 

It  should  also  be  recognized  that  the  historical 
outcome  may  have  been  a  freak  occurrence.  For 
example,  naval  warfare  modelers  would  be  hard- 
pressed  to  reproduce  the  defeat  of  the  Bismarck, 
which  entailed  a  lucky  torpedo  hit  on  the  ship’s 
rudder.  Some  have  argued  that  the  Battle  of  73 
Easting  had  many  unique  features  that  make  it  un¬ 
representative  of  likely  future  battles.  Even  if  it 
were  perfectly  representative,  it  is  only  one  case, 
and  if  other  conflicts  began  the  same  way,  they 
might  not  end  the  same  way.  The  late  military  his¬ 
torian  Trevor  DuPuy  said  in  1991  that: 

I  would  be  most  reluctant  to  take  73  Easting 
as  the  truth  of  history.  It  is  true,  but  it  is  not  nec¬ 
essarily  consistent  with  the  best  truth.  We  must 
have  many,  many  more  before  we  can  say  what 
the  lessons  are  that  have  been  learned  from  this 
(32). 

A  related  approach  would  be  to  load  SIMNET 
with  a  battle  fought  at  the  NTC  and  see  whether 
the  SIMNET  results  match  NTC  results.  The 
events  of  the  Battle  of  Seventy-Three  Easting, 
while  extraordinarily  well-documented,  are  far 
from  perfectly  known.  For  example,  it  is  not 
known  whether  the  crews  of  some  or  all  of  the 
Iraqi  tanks  were  hiding  in  bunkers  near  their  ve¬ 
hicles  when  the  U.S.  forces  opened  fire  and  could 
not  return  fire  immediately.  It  is  also  not  known 
whether  some  of  the  Iraqi  tanks  had  been  de¬ 
stroyed  by  U.S.  bombing  before  the  battle.  Final¬ 
ly,  the  outcome  of  73  Easting  may  have  been  one 
of  several  possible  outcomes.  Use  of  NTC  battles 
for  validation  would  be  preferable,  at  least  in  this 
regard,  because  the  validators  could  pick  an  NTC 
battle  that  has  been  fought  many  times  and  for 
which  there  is  a  resulting  strong  sense  of  how  it 
usually  turns  out  and  of  the  variation  in  outcomes. 

On  the  other  hand,  use  of  NTC  battles  to  vali¬ 
date  SIMNET  for  predictive  uses  would  beg  the 


See  references  (30)  and  (56)  for  other  examples. 
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question  of  whether  NTC  exercises  are  them¬ 
selves  valid  live  simulations.  There  are  doubts: 

I’m  always  concerned  about  NTC  data,  be¬ 
cause  it  isn’t  real  war,  despite  the  stress  and 
strains  and  drama  of  the  exercise.  People  know 
that  they’ll  be  going  home  that  evening  and  get¬ 
ting  a  warm,  good  meal  and  sleeping  in  a  rela¬ 
tively  warm  bed  (32). 

More  fundamentally,  model-to-model  and  sim- 
ulation-to-simulation  comparisons  are  often  dis¬ 
missed  with  the  argument  that  two  simulations 
could  agree,  and  both  could  be  wrong.  However, 
such  comparisons  are  useful,  because  if  it  turns 
out  that  two  simulations  disagree  in  an  important 
respect,  then  they  cannot  both  be  valid, and  dis¬ 
covering  this  can  be  important.  Moreover,  if  one 
of  the  simulations  was  previously  validated  and 
accredited,  it  could  be  used  as  a  proxy  for  reality, 
for  purposes  of  output  validation. 

Another,  and  very  different,  approach  to  vali¬ 
dating  SIMNET  (or  any  similar  system)  would  en¬ 
tail  detailed  examination  of  its  inner  workings. 
The  obvious  starting  point  for  this  structural  val¬ 
idation  is  the  set  of  assumptions  embodied  by  the 
model.  Some  assumptions  are  data.  Some  engi¬ 
neering  data,  such  as  armor  thicknesses  and  ve¬ 
hicle  road  speeds,  are  so  well  known  as  to  be 
unlikely  to  pose  any  problem,  but  closely  related 
performance  parameters,  such  as  armor  penetra¬ 
tion  and  vehicle  speeds  over  various  types  of  sand 
and  soil,  may  be  based  on  test-range  trials  whose 
conditions  may  or  may  not  make  them  suitable  for 
use.^^  Assumptions  that  are  not  engineering  data, 
or  perhaps  not  data  at  all,  are  harder  to  validate. 
False  assumptions  are  not  the  only  threat:  missing 
assumptions  are  much  harder  to  detect  and  their 
consequences  are  often  much  more  severe.  One 
can  satisfy  oneself  that  all  of  a  model’s  assump¬ 
tions  are  true  and  still  have  room  for  concern  that 
the  model’s  output  might  be  untrue. 


Having  examined  the  model’s  data,  a  structural 
validation  effort  would  continue  by  examining  the 
equations  and  other  relationships  that  make  the 
model  function  as  it  does.  Some  of  these,  such  as 
those  governing  the  flight  of  projectiles,  would  be 
straightforward  and  incontestable  physics  or  engi¬ 
neering  equations.  Others,  such  as  those  relating 
fires  to  smoke  and  smoke  to  obscuration,  could 
prove  at  least  as  contentious  as  the  data  used  to 
drive  them. 

In  the  case  of  simulations  such  as  SIMNET  that 
present  a  person  with  sensory  input,  a  final  step  in 
validity  checking  is  needed:  that  which  esta¬ 
blishes  that  the  sights  and  sounds  presented  to  the 
trainee  are  as  they  should  be.  If  some  submodel 
determines  that  a  given  burning  tank  should  create 
smoke  that  restricts  visibility  to  20  meters,  does 
the  model’s  synthetic  environment  in  fact  present 
objects  out  to  20  meters  and  no  farther  when  the 
user  drives  into  the  smoke  from  this  burning  tank? 

It  would  at  first  appear  that  virtual  simulation 
could  not  induce  the  perception  of  being  in  mortal 
danger.  If  this  were  true,  it  might  add  an  irresolv¬ 
able  obstacle  to  the  assessment  of  validity  of  virtu¬ 
al  simulation,  which  is  not  to  say  that  it  would 
make  inaccurate  predictions.  However,  in  some 
cases  it  may  be  possible  to  measure  the  physiolog¬ 
ical  stress  of  simulator  users  and  compare  it  to  the 
stress  of  actual  combatants.  There  are  clinical  pre¬ 
cedents:  in  two  recent  experiments  acrophobic  pa¬ 
tients  were  transported  virtually  (by  means  of  a 
head-mounted  display)  to  a  high  place  in  a  syn¬ 
thetic  environment  and  their  physiological  signs 
of  stress  were  measured  (63,82,134).  The  stress 
was  presumably  a  symptom  of  fear  (phobia). 
However,  the  only  way  to  assess  the  fearfulness, 
per  se,  of  a  subject  is  to  ask  the  subject  whether  he 
or  she  is  fearful,  as  S.L.A.  Marshall  did  for  Men 
Against  Fire  (91). 


Possibly  the  simulations  consisted  of  using  models  to  generate  predictions  based  on  erroneous,  invalid  data.  In  this  sentence,  we  use 
simulation  in  a  broad  sense,  to  include  data. 

More  likely,  and  much  more  threatening  to  validity,  tlie  performance  parameters  are  as  much  tiie  results  of  mnning  other  models  as  tliey 
are  of  testing  equipment  in  the  field.  For  a  detailed  and  seminal  treatment  of  this  problem,  see  (135). 
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The  same  could  be  done  with  users  of  simula¬ 
tors  of  real  or  proposed  weapon  systems.  The  dif¬ 
ficult  (but  not  impossible)  step  in  such  a 
comparison  would  be  making  corresponding 
measurements  on  and  assessments  of  actual  com¬ 
batants.  Some — ^perhaps  most — ^people  would 
consider  this  a  waste,  believing  it  obvous  that  real 
combat  is  more  stressful  and  fear  evoking  than 
simulated  combat,  for  most  soldiers.  Cases  of 
simulator  users  losing  the  insight  that  “it’s  not 
real”  are  so  rare  and  startling — even  amusing — 
that  they  are  recounted  as  anecdotes,  even  in  Con- 
gess,  by  proponents  of  virtual  simulation."^^ 
However,  even  if  simulators  do  not  (or  are  not  al¬ 
lowed  to)  evoke  the  same  stress  and  fear  that  com¬ 
bat  does,  the  effect  on  a  virtual  simulation’s 
predictive  accuracy  and  reliability  is  unclear,  be¬ 
cause  some  individuals  perform  better  when 
stressed  and  fearful,  while  others  perform  worse. 
Unit  leaders  endeavor  to  channel  their  soldiers’  re¬ 
sponses  to  fear  into  constructive  action  rather  than 
panic. 

Another  way  in  which  simulator  operators  may 
behave  unlike  real  combatants  is  that  they  may 
participate  in  the  same  scenario  so  many  times 
that  they  learn.  This  is  desirable  for  training  but  is 
usually  undesirable  in  research,  in  which  the  ob¬ 
jective  is  often  to  predict  how  effectively  human 
operators  will  behave  the  first  time  they  see  real 
combat.  Accruing  statistical  confidence  in  such 
predictions  requires  many  repetitions  (11,41, 
141),  but  preventing  the  operators  from  learning 
might  require  using  a  fresh  crew  for  each  repeti¬ 
tion.  An  alternative  approach  that  has  been  used  in 
the  validation  of  a  small-unit  entity-level 


constructive  simulation  (41,141)  is  randomiza¬ 
tion  of  scenarios,  to  estimate  and  account  for 
learning.^^ 

The  degree  to  which  a  model’s  output  should  be 
believed  also  depends  in  part  on  what  the  output 
is.  Models  are  to  a  considerable  degree  idealiza¬ 
tions,  and  as  such  generally  understate  the  delete¬ 
rious  effects  of  the  fog  of  wai‘,  and  portray  military 
activity  as  proceeding  more  smoothly  than  it  real¬ 
ly  does.  Similarly,  even  if  a  virtual  prototype 
works  well  in  simulation,  a  physical  system  built 
from  the  design  may  fail  because  of  some  detail  or 
phenomenon  that  was  not  modeled.  On  the  other 
hand,  if  a  virtual  prototype  functions  poorly,  it 
seems  doubtful  that  unmodeled  phenomena 
would  make  it  work  better  in  the  real  world. 

Accreditation  constitutes  official  certification 
that  a  computer  model  is  acceptable  for  a  particu¬ 
lar  purpose.  It  provides  assurance  to  users,  partic¬ 
ularly  contractors,  that  their  work  will  not  later  be 
dismissed  on  the  grounds  that  it  was  performed 
using  a  questionable  computer  model.  It  has  been 
suggested  that  accreditation  for  a  particular  pur¬ 
pose  be  extended  not  simply  to  models  but  to 
model-user  combinations  (21). 

A  problem  underlying  much  of  the  difficulty  of 
creating  valid  military  simulation  models  is  the 
lack  of  necessary  data,  and  in  general  the  lack  in 
the  U.S.  system  of  what  is  termed  military  science 
abroad  (20).  There  is  no  consensus  as  to  how  one 
validates  a  model  and  what  constitutes  a  valid 
reality  against  which  to  check  it,  though  there  is  no 
shortage  of  practical  biases  and  philosophies 
about  what  standai'ds  modeling  and  simulation 


Dave  Evans  of  Evans  and  Sutherland,  a  pioneer  in  computer  image  generation  for  cockpit  simulators  as  well  as  head-mounted  displays 
(HMDs),  has  said  that  onepilot  flying  a  Boeing  747  airplane  simulator  fainted  when  his  virtual  airplane  hit  a  ditch  on  landing.  Professor  Fredei- 
ick  Brooks,  quoting  him,  said  that  nothing  so  dramatic  had  happened  to  a  trainee  wearing  an  HMD  (10). 

43  por  example,  if  a  simulation  experiment  were  conducted  to  assess  whether  tankunits  would  be  more  effective,  in  a  certain  scenario,  if  tlieir 
tanks  were  equipped  with  a  new  radio  system,  the  crews  of  10  units  might  participate  in  10  exercises  each — 5  exercises  using  the  simulated 
inventory  radios  and  5  exercises  using  the  simulated  proposed  radios.  The  order  in  which  each  unit  uses  inventory  radios  and  proposed  radios 
should  be  chosen  randomly,  so  that,  for  example,  crews  do  not  all  use  the  inventory  radios  5  times  while  warming  up  and  then  the  proposed 
radios  5  times  after  having  improved  their  proficiency — an  improvement  that  careless  analysis  might  attribute  solely  to  advantages  of  the  pro¬ 
posed  radios.  More  sophisticated  experiments  and  analysis  could  estimate  the  rate  of  learning  and  extrapolateperformance  witli  eachradio  back 
to  estimate  cold  (i.e.,  unrehearsed)  performance  witli  each  radio  or  forward  to  estimate  performance  with  each  radio  after  much  practice  and 
rehearsal. 
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should  meet.  Lack  of  standards  for  models  inhib¬ 
its  their  verification  and  use.  A  program  manager 
trying  to  select  an  appropriate  model  to  use  to  sup¬ 
port  his  program  “finds  it  very  difficult  to  locate 
unbiased  help  in  deciding  what  model  to  choose, 
what  data  [are]  acceptable,  and  what  models  have 
been  successful  in  the  past.  Each  program  manag¬ 
er  is,  to  a  great  extent,  left  on  his  own  to  make 
these  decisions  and  then  defend  them  against  a 
host  of  organizations  which  are  chartered  to  chal¬ 
lenge  such  decisions”  (101,  p.  36).  Models  tend  to 
be  biased  toward  fulfilling  the  needs  of  a  particu¬ 
lar  military  service  or  defense  agency.  They  will 
represent  with  high  fidelity  the  details  of  interest 
to  a  particular  service  but  not  those  of  interest  to 
other  services.  Different  communities  have  con¬ 
tradictory  requirements  for  models.  Models  de¬ 
veloped  by  the  acquisitions  community,  for 
example,  are  traditionally  not  accepted  by  test  and 
evaluation  agencies.  Current  practice  for  test  and 
evaluation  personnel  is  to  assume  that  acquisition 
models,  which  are  developed  by  the  contractors  of 
the  system,  are  inherently  biased  (101 ,  p.  2).  In  the 
training  community,  the  validity  of  SIMNET  ex¬ 
ercises  is  questioned,  since  they  do  not  meet  tradi¬ 
tional  training  measures  (201,  pp.  14-15). 

One  interesting  contrast  between  the  field  of 
defense  modeling  and  other  similar  fields,  e.g. 
econometrics,'*^  is  the  lack  of  a  tradition  of  vali¬ 
dating  models  routinely  (21,  p.  30).  Peer  review 
would  seem  to  present  an  ideal  solution  to  the  dif¬ 
ficulties  posed  by  the  field  of  defense  modeling, 
but  peer-reviewed  journals  cater  to  an  academic 
side  of  the  field  that  is  more  that  usually  detached 
fi'om  the  real-world  version.  (This  split  is  doubt¬ 
less  exacerbated  by  security  considerations.)  Ar¬ 
ticles  appearing  in  these  journals  often  have  a 
more  judgmental  character  than  articles  in  scien¬ 


tific  journals.  The  field  of  defense  modeling  com¬ 
bines  empirical,  historical,  mathematical,  and 
judgment  in  a  mix  likely  to  sow  disesteem  among 
peers:  the  more  subjective  aspects  create  plenty  of 
room  for  disagreement,  while  the  ability  to  point 
to  a  few  facts  allows  the  disputants  to  feel  that 
their  opponents  are  not  dealing  with  reality. 

It  is  in  this  context,  then,  that  one  must  consider 
short-term  proposals  to  accomplish  validation  in 
part,  or  entirely,  by  peer  review  of  the  work  or  by 
the  certification  or  licensing  of  the  workers.  Work¬ 
ers  fear  that  such  review,  certification,  or  licensing 
by  peers  would  at  best  institutionalize  back-stab¬ 
bing  and  at  worst  foster  the  creation  and  self-per- 
petuation  of  a  one-viewpoint  orthodoxy. 

In  the  long  run,  peer  review  could  perhaps  be 
made  to  work  if  a  healthier  combat  modeling  cul¬ 
ture  develops.  At  present  such  a  culture  is  hard  to 
identify  because  of  the  security  classification  sur¬ 
rounding  a  considerable  portion  of  the  work,  the 
disparate  backgrounds  of  workers  in  the  field,  the 
near-total  lack  of  a  full-fledged  academic  curricu¬ 
lum  related  to  combat  modeling,^^  and  the  distaste 
with  which  some  view  the  organized  study  of  how 
to  kill  people  in  large  numbers.  These  barriers 
seem  difficult  to  surmount,  and  lead  to  the  discus¬ 
sion  of  standards  in  the  following  section.  It  is  im¬ 
portant  to  note,  however,  that  one  form  of  peer 
review  exists  at  present  and  appears  to  work — ^in¬ 
ternal  peer  review.  Through  a  conscientious  pro¬ 
gram  of  internal  peer  review,  some  organizations 
devoted  to  defense  modeling  and  simulation  do 
manage,  according  to  their  own  view,  to  enhance 
the  quality  of  their  products. 

A  number  of  recent  efforts  to  improve  valida¬ 
tion  of  M&S,  including  some  used  for  DIS,  are 
noted  in  the  following  subsections. 


^  Even  in  econometrics,  the  tltting  of  a  model  to  data  is  more  common  than  is  the  statistical  testing  of  the  resulting  model  on  the  basis  of 
subsequent  data,  accounting  for  degrees  of  freedom  (i.e.,  the  number  of  ways  in  which  the  model  can  be  made  consistent  with  tlie  peculiarities  of 
the  data  to  which  it  was  fit,  rather  than  to  basic  trends  in  the  data).  Complex  models  developed  by  academic  economists  are  seldom  validated 
(83,84), 

OTA  recognizes  tliat  several  first-rate  universities  offer  courses  of  study  and  research  in  defense  policy  analysis,  but  tliese  programs  are 
small  and  cast  a  wide  net.  They  often  combine  political  science  and  defense  modeling  in  a  single  course. 
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I  Army  Pamphlet  5-11 

The  Anny’s  pamphlet,  Verification,  Validation, 
and  Accreditation  of  Army  Models  and  Simula¬ 
tions,  issued  in  1993,  (174)  is  the  most  detailed  of¬ 
ficial  plan  for  VV&A  yet  released  by  a  DoD 
component.  More  recently,  the  Marine  Corps  is¬ 
sued  a  VV&A  plan,  and  the  other  services,  and 
DoD,  have  drafted  plans. 

I  SEES  2.0  Accreditation  Exercises 

From  1993  (41)  through  1995  (141)  Lawrence 
Livermore  National  Laboratory’s  Conflict  Simu¬ 
lation  Laboratory  (LLNL  CSL)  has  documented 
a  series  of  force-on-force  (FOF)  exercises  (live 
simulations).  They  were  conducted  in  October 
1992  at  the  Wurtsmith  Air  Force  Base  Weapon 
Storage  Area  to  “accredit”  (as  well  as  validate) 
the  Security  Exercise  Evaluation  System  version 
2.0  (SEES  2.0).  SEES  2.0  is  a  multiuser,  interac¬ 
tive,  entity-level  constructive  simulation,  that  can 
simulate  the  behavior  of  individuals  in  small  units 
such  as  infantry  squads,  security  police  patrols, 
special  forces  units,  or  terrorist  groups,  under  the 
supervision  of  squad  or  team  leaders.  Although 
SEES  2.0  is  neither  a  DIS  or  a  virtual  simulation, 
the  method  used  by  CSL  to  inform  accreditation 
eould  be  used  to  validate  distributed  interactive 
virtual  simulations.  We  briefly  summarize  the 
SEES  accredition  study  here. 

The  LLNL  CSL  developed  SEES  2.0  for  the 
Air  Force  Security  Police  and  the  Defense  Nuclear 
Agency  for  use  in  evaluating  the  physical  security 
of  weapon  storage  areas  and  other  facilities.  It  was 
developed  from  the  Urban  Combat  Computer  As¬ 
sisted  Training  System  and  later  evolved  into  the 
Joint  Tactical  Simulator. 

The  accreditation  study  defined  two  quantita¬ 
tive  measures  of  performance  that  were  predicted 
by  the  SEES,  relevant  to  the  application,  and  mea¬ 
surable  in  live  simulations.  One  measure  was  the 
friendly-to-hostile  force  ratio,  which  varied  as  the 
live  or  constructive  simulation  progressed.  The 
other  measure  was  the  time  (after  the  beginning  of 
the  simulation)  at  which  key  events  occurred,  such 
as  the  transport  of  a  stolen  weapon  through  the 
weapon  storage  area  security  perimeter.  For  each 


of  several  conditions  (scenarios),  several  attacks 
were  simulated.  In  the  terminology  of  experimen¬ 
tal  design,  this  provided  for  blocking  and  replica¬ 
tion. 

SEES  2.0  is  a  stochastic,  Monte  Carlo  simula¬ 
tion.  In  different  SEES  simulations  (as  in  live  sim¬ 
ulations)  starting  from  the  same  condition, 
different  things  happen  in  general  and  to  the  mea¬ 
sures  of  performance  in  particular.  Several  SEES 
simulations  and  several  live  simulations  were 
conducted  for  each  scenario,  to  increase  the  statis¬ 
tical  confidence  with  which  one  could  conclude 
that  SEES  was  a  valid  model  of  the  force-on-force 
exercises,  or  to  reduce  (i.e.,  improve)  the  statisti¬ 
cal  significance  with  which  one  could  conclude 
that  performance  measures  of  SEES  simulations 
were  statistically  different  from  those  of  the  FOF 
exercises.  The  SEES  and  FOF  results  were 
comparedusing  a  two-sample  Kolmogorov-Smir- 
nov  test.  (See  box  8.)  At  most  times  during  the 
scenario,  one  could  not  distinguish  the  SEES 
force  ratios  from  the  FOF  force  ratios  with  less 
than  a  10-percent  chance  of  error.  However,  there 
was  one  time  at  which  the  SEES  force  ratios  could 
be  distinguished  fi'om  the  FOF  force  ratios  with 
less  than  a  10-percent  chance  of  error,  implying 
that  SEES  did  not  accurately  predict  what  the  live 
forces  did  at  that  time  in  the  live  simulation.  Anal¬ 
ysis  of  simulation  data  indicated  that  SEES  pre¬ 
dicted  implausible  individual  and  unit  behavior  in 
a  certain  ambush  situation.  This  conclusion  led  to 
subsequent  adjustment  (tuning)  of  the  SEES  mod¬ 
el. 

The  accreditation  study  report  acknowledged 
that  several  issues  of  validity  could  be  raised:  one 
is  whether  the  live  forces,  who  1)  used  MILES 
equipment  instead  of  actual  weapons,  2)  were 
obliged  to  observe  Occupational  Health  and  Safe¬ 
ty  Administration  regulations,  and,  3)  because  of 
instrumentation  limitations,  were  required  to  op¬ 
erate  in  daylight  (contrary  to  doctrine  and  train¬ 
ing)  in  most  replications,  could  be  considered 
proxies  for  real  forces  operating  under  operational 
conditions.  The  scaling  of  scenario  time  to 
compare  the  pace  of  action  in  the  live  exercises 
with  the  faster  pace  in  SEES  runs,  and  the  use  of 
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BOX  8:  Use  of  the  Two-Sample  Kolmogorov-Smirnov  Test  in  Validation 


The  two-sample  Kolmogorov-Smirnov  (K-S)  tesf  is  a  statistical  procedure  that  can  be  used  to 
decide  whether  two  sets  of  data  (for  example,  one  set  from  a  constructive  simulation,  the  other  from  a 
live  simulation  or  from  history)  are  statistically  equivalent, 

Specifically,  the  two-sample  K-S  test  is  a  statistical  procedure  for  deciding  whether  two  groups 
(samples)  of  numbers,  each  selected  randomly  (and  independently,  with  replacement)  from  a  larger 
group  (a  population)  of  numbers,  were  selected  from  the  same  population  or,  alternatively,  from  different 
populations. 

For  example,  in  one  of  the  K-S  tests  performed  in  the  SEES  V2.0  accreditation  project,  one  sam¬ 
ple  (called  the  simulation  sample)  was  a  set  of  40  Red/Blue  (i.e.,  hostile-to-friendly)  surviving  force  ra¬ 
tios  at  two  minutes  after  the  commencement  of  hostilities,  from  40  simulations  of  a  scenario  (called  Sce¬ 
nario  3)  using  SEES  V2.0,  a  constructive  simulation  with  human  participation.  The  other  sample  (called 
the  experimental  sample)  was  a  set  of  20  Red/Blue  surviving  force  ratios  at  two  minutes  after  the  com¬ 
mencement  of  hostilities,  from  20  force-on-force  (FOF)  exercises  with  the  same  scenario.  On  the  basis 
of  a  two-sample  K-S  test,  the  authors  of  the  project’s  final  repoif  concluded,  with  a  statistical  signifi¬ 
cance  of  about  0.0281,  that  the  two  samples  of  force  ratios  were  drawn  from  different  populations— i.e., 
that  the  surviving  force  ratios  predicted  by  SEES  V2.0  were  not  the  same,  statistically,  as  the  surviving 
force  ratios  from  the  live  simulation.  The  phrase  “with  a  statistical  significance  of  about  0.0281”  means 
“with  a  probability  of  error  no  greater  than  about  0,0281”  (i.e.,  about  3  percent).  It  was  later  determined' 
that  this  discrepancy  arose  In  part  because  of  “the  Inadequate  modeling  in  SEES  of  human  factors  for 
ambush  situations. 

The  qualification,  “ _ about refers,  in  part,  to  the  fact  that  the  authors  used  an  approximate 

version  of  the  K-S  test  that  is  accurate  for  comparing  very  large  samples  (i.  e.,  samples  containing  data 
from  many  replications  of  a  simulation  or  experiment)  but  is  not  necessarily  accurate  for  comparing 
samples  of  20  or  40  numbers.  An  assessment  of  the  accuracy  for  the  sample  sizes  used  in  the  accredi¬ 
tation  project  was  planned  but  was  never  conducted  because  of  resource  limitations. 

OTA  also  uses  the  qualification,  “...  about..,  ”  to  refer  to  uncertainties  introduced  by  other  prac¬ 
tices.  For  example,  as  noted  above,  the  K-S  test  may  only  be  properly  applied  to  samples  selected 


the  asymptotic  (large-sample  size)  version  of  the 
Kolmogorov-Smirnov  test  for  the  comparisons 
were  also  flagged  for  discussion. 

I A2ATD  Validation  Experiments 

In  1994  the  Army  conducted  the  first  of  six  experi¬ 
ments  planned  to  validate  BDS-D  simulators, 
SAP  (called  SAPOR),  and  the  BDS-D  simulation 
(simulators  and  SAP  operating  together).  The 
VV&A  effort  is  funded  by  the  Army’s  BDS-D 
program  and  is  conducted  as  part  of  the  Army-  and 
ARPA-funded  Anti- Armor  Advanced  Technolo¬ 
gy  Development  program.  The  VV&A  effort  is  in¬ 
tended  to  address  a  deficiency  of  particular 
interest  to  Congress:  “The  current  BDS-D  simu¬ 


lation  capabilities  are  not  adequate  for  evaluations 
to  support  acquisition  decisions.  . .  A2ATD.  .  . 
was  initiated  with  the  goal  of  maturing  DIS  as  a 
credible  evaluation  tool  to  support  acquisition  de¬ 
cisions”  (11).  Box  9  describes  the  A2ATD  in 
greater  detail. 

I  Joint  Staff  Instruction  on  VV&A 

On  January  12,1995,  the  Vice  Director  of  the  Joint 
Staff  issued  Joint  Staff  Instruction  8104.01,  Veri¬ 
fication,  Validation,  and  Accreditation  of  Joint 
Models  and  Simulations  (183).  Modeled  on  Army 
Pamphlet  5-11,  it  specifies  general  VV&A  proce¬ 
dures  for  use  by  the  Joint  Staff. 
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BOX  8:  Use  of  the  Two-Sample  Kotmogorov-Smirnov  Test  in  Validation  (Cont’d.) 


randomly  and  independently,  with  replacement,  from  a  population.  In  the  case  of  the  simulation  sample 
or  the  experimental  sample,  the  population  refers  to  all  the  force  ratios  that  would  occur  if  the  exercise 
were  repeated  an  infinite  number  of  times,  in  each  case  under  the  same  conditions  (including  partici¬ 
pant  experience).  Random  and  independent  selection,  with  replacement,  implies  that  each  possible 
force  ratio  should  have  the  same  probability  of  occurring  regardless  of  the  replication  number—e.g.,  it 
should  have  the  same  probability  of  occurring  In  the  first,  second,  and  twentieth  replication.  This  implies 
that  there  should  be  no  learning  by  Red  or  Blue  forces,  over  the  course  of  those  successive  simula¬ 
tions,  in  any  way  that  would  have  an  influence  on  the  probabilities  of  force  ratios  at  two  minutes  after  the 
commencement  of  hostilities.  In  order  “to  reduce  [the]  learning  curve”  in  the  force-on-force  exercises, 
the  SEES  accreditation  project  allowed  the  Red  and  Blue  forces  38  “free  plays,  ”  in  addition  to  the  40 
“record  events”'‘(20  for  each  of  2  scenarios),  in  the  force-on-force  exercises.  Free  play  was  also  al¬ 
lowed  in  the  later  SEES  V2.0  simulations.  In  spite  of  this,  the  SEES  participants,  “without  coaching, 
learned  the  same  successful  tactics  as  [did  the]  FOF  [exercise]  participants. 

The  experimental  and  simulation  samples  for  other  times  after  the  commencement  of  hostilities 
(at  least  until  eight  minutes)  were  in  better  agreement.  For  times  at  30-second  intervals  from  the  start 
until  seven  minutes  later  (except  for  two  minutes  after  start),  the  experimental  and  simulation  samples 
were  not  so  different  statistically  that  the  K-S  test  could  reject  the  hypothesis  that  they  were  drawn  from 
the  same  population  with  a  statistical  significance  better  (i.e.,  less)  than  0.10.  In  fact,  at  4.5  minutes 
after  the  commencement  of  hostilities  the  experimental  and  simulation  samples  were  so  similar  that  the 
hypothesis  that  they  were  drawn  from  the  same  population  could  not  be  rejected  with  statistical  signifi¬ 
cance  better  than  0.9853, 


’D.A.  Darling,  'The  Kotmogorov-Smirnov,  Cramer-von  Mises  Tests,  ”  Annals  of  Mathematical  Statistics,  28:823-838, 1957. 
*G.  Friedman  and  R.  Toms,  Security  Exercise  Evacuation  System  (SEES)  V2.0,  Accreditation  Project  Final  Report,  February 
1 993,  FOR  OFFICIAL  USE  ONLY, 

®R.M.  Toms,  “Results  of  a  Verification,  Validation,  and  Accreditation  Excercise,"  briefing,  presented  at  the  Office  of  Technology 
Assessment  Workshop  on  Issues  in  Combat  Model  Validation,  Washington,  DC,  Feb.  8, 1995. 

SOURCE:  Office  of  Technology  Assessent,  1995, 


I  TRADOC  Research  on  DiS  Validation 

The  Army’s  Training  and  Doctrine  Command 
(TRADOC)  has  been  sponsoring  conceptual  work 
on  DIS  validation.  One  study,  by  the  RAND  Cor¬ 
poration,  which  was  to  consider  alternative  ap¬ 
proaches  to  the  problem,  is  nearly  complete  (26). 
It  elaborates  ideas  expressed  in  earlier  RAND 
work  on  validation  (21,22,64),  focusing  on  DIS 
and  emphasizing  that  there  are  a  number  of  non- 
predictive  uses  of  DIS  for  which  comparison  of 
model  predictions — or  DIS  outcomes,  construed 
as  predictions — to  historical  or  future  real-world 


outcomes  is  neither  necessary  nor  appropriate  for 
validation.  For  example,  if  a  system  of  DIS  equip¬ 
ment,  software,  and  databases  is  to  be  used  for 
training,  then  validation  should  seek  to  assess  the 
degree  to  which  trainees  have  learned  the  desired 
conditional  behaviors  as  a  result  of  the  train- 

•  4fi 

ing. 

Predictive  uses  are  classified  into  various  sub¬ 
categories  of  weakly  predictive  uses  and  strongly 
predictive  uses,  the  former  denoting  predictions 
with  considerable  uncertainty,  such  as  theater 
combat  simulations,  and  tlie  latter  denoting  pre- 


Tliis  begs  the  queiition;  why  are  those  conditional  behaviors  believed  tO  be  good  or  optimal’? 
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BOX  9:  Anti-Armor  Advanced  Technology  Development  (A2ATD) 


The  Anti-Armor  Advanced  Technology  Development  (A2ATD)  program  was  undertaken  by  the 
Army  and  DoD  as  a  whole,  with  the  goal  of  Improving  DIS  applications  enough  to  be  credible  as  an 
evaluation  tool  to  support  acquisition  decisions.  The  purpose  of  the  A2ATD  is  to  develop  and  demon¬ 
strate  a  verified,  validated,  and  accredited  DIS  capability  to  support  anti-armor  weapon  system  virtual 
prototyping,  concept  formulation,  requirements  definition,  effectiveness  evaluation,  and  mission  area 
analysis  on  a  combined  arms  battlefield  at  the  battalion  task  force  or  brigade  level. 

The  A2ATD  program  has  four  technical  objectives: 

■  demonstrate  DIS  as  an  evaluation  tool  and  verify,  validate,  and  accredit  simuiators  used  in  A2ATD 
experiments,  semiautomated  forces,  and  the  Battlefield  Distributed  Simulation-Developmental 
(BDS-D)  simulation; 

■  develop,  demonstrate,  and  document  techniques  and  analytical  tools  to  evaluate  the  causes  of 
simulation  outcomes: 

■  link  one  or  more  constructive  models  such  as  Janus  to  DIS;  and 

■  demonstrate  upgraded  virtual  prototypes  of  M1A2  Abrams  tanks,  M2A3  and  M3A3  Bradley  ar¬ 
mored  fighting  vehicles,  non-line-of-sight  (NLOS)  and  llne-of-sight  antitank  (LOSAT)  weapons,  and 
virtual  prototypes  to  be  developed— of  the  Apache  and  Comanche  helicopters,  the  Enhanced  Fi¬ 
ber  Optic  Guided  Missile  (EFOGM),  AGS,  JAVELIN,  and  Hunter. 

In  FY  1993  VV&A  of  simulated  and  semiautomated  forces  and  development  of  analytical  tools  to 
support  the  evaluation  of  causes  of  simulation  outcomes  began,  with  the  goal  of  providing  the  founda¬ 
tion  for  a  series  of  six  experiments  In  FY  1994-1996.  The  first  experiment,  completed  in  September 
1994,  replicated  two  Ml  A2  Initial  Operational  Test  and  Evaluation  (lOT&E)  vignettes  to  compare  the  re¬ 
sults  of  virtual  simulation  with  those  of  live  simulation  and  to  compare  constructive  simulation  with  live 
and  virtual  simulation.  Four  experiments  planned  for  FY  1995  will  use  BDS-D  simulation  of  the  effects  of 
heavy  force  modernization  In  a  Southwest  Asia  scenario  and  demonstrate  the  linkage  of  Janus  to  BDS- 
D,  BDS-D  and  ModSAF  VV&A  will  be  systematically  expanded  by  addition  of  the  following  simulators: 
Apache,  Comanche,  M1A2,  M1A2  firing  STAFF,  LOSAT,  and  NLOS.  One  experiment  will  be  conducted 
in  FY  1996,  a  Rapid  Fire  Projection  Initiative  scenario  with  AGS,  LOSAT,  EFOGM,  JAVELIN,  and  Hunter 
simulators.  Experiments  1  ,2,  and  4  will  be  local-area  network  experiments  conducted  at  the  Mounted 
Warfare  Test  Bed  (MWTB).  Experiments  3,  5  and  6  will  be  wide-area  network  experiments.  Experiment  3 
will  link  the  MWTB,  Land  Warner  Test  Bed  (LWTB),  and  the  BDS-D  site  atTACOM.  Experiment  5  will  link 
the  MWTB,  LWTB,  and  the  Aviation  Test  Bed  (AVTB).  Experiment  6  will  link  the  MWTB,  LWTB,  and  BDS- 
D  site  at  MICOM. 

The  A2ATD  experiment  analysis  cycle  will  proceed  as  follows.  The  scenario  vignette  and  perfor¬ 
mance  data  will  feed  into  BDS-D  and  the  Combat  Arms  Task  Force  Engagement  Model  (CASTFOREM). 
BDS-D  simulations  will  be  run  with  verified,  validated,  and  accredited  ModSAF  and  simulators  (level  2 
CIGs)  in  a  level  2  environment  with  level  2  DIS  standards.  BDS-D  simulation  runs  will  be  made  with  and 


dictions  with  little  uncertainty,  such  as  some  engi¬ 
neering  simulations.  The  draft  report  concludes 
that  use  of  subjective  judgement  may  be  necessary 
for  weakly  predictive  uses,  and  it  notes  the  impor¬ 
tance  of  describing  and  quantifying  the  uncertain¬ 
ties  in  the  predictions. 


I  MORS  SIMVAL  '94  Symposium 

In  September  1994  the  Military  Operations  Re¬ 
search  Society  held  SIMVAL  ’94,  die  fourth  in  a 
series  of  mini-symposia  and  workshops  designed 
ty  address  the  problem  of  simulation  validation. 
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BOX  9;  Anti-Armor  Advanced  Technology  Development  (A2ATD)  (Cont’d.) 


without  simulators  to  provide  the  basis  for  comparing  SAF  and  simulator  behaviors  and  simulation  out¬ 
comes.  ModSAF  behavior  algorithms  will  be  changed  as  appropriate  for  subsequent  experiments. 
BDS-D  simulation  outcomes  will  also  be  compared  to  CASTFOREM  outcomes  to  determine  difference- 
sand  reasons  for  differences.  CASTFOREM  algorithm  changes  and  runs  may  be  made  to  try  to  bring 
the  outcomes  into  better  agreement  with  BDS-D  simulation  outcomes.  CASTFOREM  runs  will  be  made 
for  the  full  scenario  with  the  version  of  CASTFOREM  that  produces  the  best  agreement  with  BDS-D  sim¬ 
ulation  results.  Results  of  the  BDS-D  simulation  and  CASTFOREM  runs  will  be  analyzed  to  evaluate  anti¬ 
armor  weapons’  contribution  to  force  effectiveness  and  to  provide  the  basis  for  BDS-D  simulation  VV&A. 

ModSAF  physical  algorithms  for  vulnerability  target  acquisition,  delivery  accuracy,  mobility,  and 
rate  of  fire  were  validated  as  conforming  to  Army  standards,  and  system  performance  was  consistent 
with  the  M1A2  simulator.  Behavioral  algorithms  were  validated  as  being  able  to  represent  armor  tactics 
and  doctrine.  ModSAF  was  benchmarked  for  Experiment  1  against  lOT&E  and  CASTFOREM. 

DIS  analytical  tools  were  developed  to  support  simulator  and  ModSAF  VV&A  and  to  evaluate 
causes  of  simulation  outcomes.  A  quick-look  analysis  capability  was  developed  to  permit  review  of  ex¬ 
periment  outcomes  within  30  minutes  of  trial  completion.  Protocol  data  units  were  added  and  post-pro¬ 
cessors  were  developed  to  analyze  the  causes  of  simulation  outcomes  for  the  spectrum  of  measures  of 
effectiveness,  performance,  and  behavior  specified  in  the  evaluation  plan.  Each  element  of  the  network 
was  integrated  to  insure  that  it  could  send  and  receive  protocol  data  units.  The  network  was  also  exer¬ 
cised  to  insure  that  it  had  sufficient  throughput  to  render  real  time  images  for  the  scenario. 

Detailed  evaluation  and  test  plans  were  prepared  and  troops  were  trained  before  the  experiment. 
Pilot  tests  were  run  to  insure  that  the  experiment  could  be  executed  and  data  could  be  collected  and 
analyzed  using  DIS  analytical  tools.  CASTFOREM  was  run  prior  to  the  experiment  to  fine  tune  the  sce¬ 
nario  and  benchmark  ModSAF. 

Forty-eight  trials  were  run  over  a  12-day  period.  Twenty-four  trials  were  run  for  each  vignette  (12 
trials  with  manned  simulators  and  12  trials  with  ModSAF  only).  The  two  vignettes  run  were  a  hasty  at¬ 
tack  (14  M1A2  tanks  attacking  4tanks  and  3  BMPs’)  and  a  hasty  defense  (14  Ml A2  tanks  defending 
against  26  tanks  and  1  BMP).  Four  Ml  A2  simulators  were  used  in  the  manned  simulator  trials.  To  mini¬ 
mize  crew  learning  effects  the  scenario  and  platoon  location  were  randomized. 

The  first  A2ATD  Experiment  developed  and  demonstrated  the  first  integrated  DIS  capability  using 
2.0.3+  protocols,  high-fidelity  environment,  next-generation  image  generators,  validated  simulators  and 
ModSAF,  and  DIS  analytical  tools  to  support  VV&A  and  analysis  of  experiment  outcomes.  As  a  by-prod¬ 
uct  of  the  first  experiment,  the  MWTB  was  modernized  and  W&A  tools  and  procedures  are  being  of¬ 
fered  as  DIS  VV&A  guidelines. 


'  A  BMP  IS  a  type  of  armored  vehicle  originally  produced  in  the  Union  of  Soviet  Socialist  Republics 

SOURCE:  W.J.  Brooks,  “Anti-Armor  Advanced  Technology  Demonstration  (A2ATD),'’  Phalanx,  December  1994;  available  online  at 
<URL:gopher//triton.dmso.mil.:70/11/twg/mors/phalanx/a2atd.txt> 


The  objective  of  SIMAL  ’94  was  to  address  the  units  for  V&V”  (207).  The  implication  is  that  the 
question:  “How  much  validation  is  enough?”  The  MORS  group  could  not  find  a  way  to  determine 
group  agreed  that  “there  were  no  quantifiable 
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how  much  validation  there  is,  or  how  much  is 
enough.^^ 

Afterwards,  the  SIMVAL  Senior*  Advisory 
Group  met  and  decided  to  use  the  output  of  the 
workshop  to  draft  a  set  of  VV&A  templates.  The 
draft  guidelines  will  be  posted  on  an  electronic 
bulletin  board  system  for  review,  comment,  and 
revision.  When  it  appears  that  a  consensus  has 
been  reached,  the  revised  guidelines  will  be  pub¬ 
lished.  No  additional  SIMVAL  workshops  will  be 
planned  until  the  M&S  community  has  had  a 
chance  to  use  the  draft  guidelines  in  case  studies 
(207). 

I  DoD  Modeling  and  Simulation 
Master  Plan 

In  January  1995  the  Under  Secretary  of  Defense 
for  Acquisition  and  Technology  began  formal 
coordination  of  the  Final  Draft  of  the  DoD  Model¬ 
ing  and  Simulation  Master  Plan  (195).  A  complex 
and  important  document,  it  lays  out  a  vision, 
goals,  and  responsibilities  for  VV&A  and  other 
tasks  required  (153)  for  management  of  Defense 
models  and  simulations.  Specifically,  it  requires 
a  DoD  Instruction  on  VV&A  to  be  prepared — and 
one  has  been  drafted  (154).  It  echoes  the  question, 
“How  much  validation  is  enough?”  as  an  unre¬ 
solved  issue. 

I  Methodology  Handbook 
for  VV&A  of  DIS 

DMSO  has  sponsored  the  drafting  of  a  methodol¬ 
ogy  handbook  for  VV&A  of  DIS  (175).  It  is  aimed 
at  DIS  exercise  managers.  A  companion  docu¬ 
ment,  the  DIS  W&A  Implementation  Guide,  is 
being  developed  to  describe  in  detail  the  actions 
required  to  accomplish  each  step  identified  in  this 
VV&A  handbook.  If  done  weU,  the  DIS  W&A 
Implementation  Guide  could  serve  as  the  much 


needed  key  to  quantifying  validation  so  the  re¬ 
quirements  could  specify  how  much  is  enough. 

I  OTA  Workshop  on  Validation 

OTA  hosted  a  workshop  on  issues  of  validating 
combat  models  on  February  8,  1995.  OTA’s  pur¬ 
pose  was  to  hear  views  and  collect  information  for 
possible  inclusion  in  the  report  of  this  assessment. 
Some  of  the  material  presented  is  summarized  in 
this  background  paper — e.g.,  in  the  section 
“SEES  2.0  Accreditation  Exercises”  above  and  in 
the  following  section. 

I  Validation  of  CCTT  for  Training 

The  Project  Manager  (PM  CATT)  of  the  Army’s 
Combined  Arms  Tactical  Trainer  project  (CATT) 
has  been  conducting  a  vigorous  effort  to  validate 
the  Close  Combat  Tactical  Trainer  (CCTT),  which 
is  the  first  of  several  DIS-based  training  systems 
to  be  developed  under  the  CATT  funding  and 
management  umbrella.  PM  CATT  recognizes  that 
because  CCTT  is  to  be  used  for  training,  CCTT 
validation  should  seek  to  assess  the  degree  to 
which  trainees  learn  desired  conditional  behaviors 
(i.e.,  different  behaviors  in  various  situations)  as 
a  result  of  CCTT  training.  Documenting  the  be¬ 
haviors  desired  in  various  tactical  situations  has 
been  a  major  effort,  involving  detailed  research 
into  hundreds  of  Army  trainmg  manuals,  field 
manuals  on  doctrine,  and  other  documents.  How¬ 
ever,  this  process  is  a  necessary  step  in  the  mea¬ 
surement  of  the  effectiveness  of  training  and  of 
training  transfer — i.e.,  the  transfer  of  behaviors 
learned  in  a  simulator  to  a  real  vehicle. 

OTA  notes  that  validation  of  CCTT  (or  any  oth¬ 
er  program)  for  particular  training  uses  does  not 
validate  it  for  predictive  uses,  such  as  predicting 
the  force  multiplication  that  some  proposed  com¬ 
mand  and  control  system  would  provide  to  a  tank 
battalion  under  specified  circumstances.  Such  a 


NO  TAG  OTA  partly  disagrees :  the  SEES  accreditation  study  illustrates  how  specific  measures  of  performance  can  be  defined  and  how  statis- 
tical  tests  can  determine  the  statistical  significance  at  which  one  cannot  distinguish  simulation  outputs  from  actual  outcomes  in  history,  future 
history,  or  some  set  of  live  simulations  considered  to  be  a  proxy  for  future  history.  Of  course,  details  of  implementing  such  a  comparison  will 
vary  from  case  to  case,  and  how  little  statistical  significance  to  require  ( “how  much  validation  is  enough?”)  remains  a  subjective  matter,  gener¬ 
ally  implying  a  value  judgment. 
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FIGURE  7:  Simulation  Endorsement  Process  Proposed  for  Acquisition  Applications 


SOURCE:  DoD  Executive  Council  for  Modeling  and  Simulation 


use  would  be  a  misappropriation  (4)  of  CCTT 
models  and  simulations,  unless  they  are  validated 
for  the  predictive  use. 

I  Cost/Training  Effectiveness  Analysis 

A  recent  report  by  DoD’s  Office  of  the  Under  Sec¬ 
retary  of  Defense  for  Personnel  and  Readiness 
noted  that  an  early  evaluation  of  CCTT  “Quick- 
start”  modules  will  be  the  first  major  DoD  study 
to  quantify  the  training  benefits  of  advanced  dis¬ 
tributed  simulation  (192).  A  project  to  improve 
Cost/Training  Effectiveness  Analysis  is  now  in 
progress  (in  Phase  2)  at  the  University  of  Central 
Florida’s  Institute  for  Simulation  and  Training;  it 
is  sponsored  by  the  Office  of  the  Assistant  Secre¬ 
tary  of  Defense  (Force  Management  and  Person¬ 
nel). 


I  Simulation  Endorsement  Process 

At  the  March  22,  1995,  meeting  of  DoD’s  Execu¬ 
tive  Council  for  Modeling  and  Simulation  (EX- 
CIMS),  Captain  James  HoUenbach,  USN, 
Director  of  DMSO,  briefed  the  EXCIMS  on  a  pro¬ 
posed  simulation  endorsement  process  that  would 
involve  the  EXCIMS  in  the  joint  acquisition  pro¬ 
cess  as  a  simulation  endorsing  agency.  Following 
discussion,  the  EXCIMS  agreed  to  test  the  process 
in  the  context  of  the  Joint  Advanced  Strike 
Technology  (JAST)  Program,  which  volunteered 
to  test  the  process.  Based  on  the  results  of  this  test, 
the  EXCIMS  will  decide  whether  it  will  serve  as 
a  simulation  endorsing  agency  (180).  The  process 
(see  figure  7  (67))  is,  in  essence,  a  process  that  the 
EXCIMS  could  use  to  assure  itself,  and  to  try  to 
assure  the  Defense  Acquisition  Board  (DAB),  that 
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BOX  10:  Protocol  Data  Units  (PDUs) 


The  DIS  protocol  specifies  types  and  formats  of  Protocol  Data  Units  (PDU)s,  which  are  packets 
(physically  speaking)  or  datagrams  (logically  speaking)  of  simulation  data  transmitted  from  one  simula¬ 
tor  to  any  or  ail  other  simulators  in  a  distributed  simulation.  Various  kinds  of  computers  and  simulators 
can  communicate  simulation  data  to  one  another  by  using  standard  PDUs  Most  PDUs  defined  by  the 
current  DIS  protocol  standard  are  variations  of  the  User  Defined  Protocol  (UDP)  datagrams  used  by 
some  computers  on  the  Internet. 

The  DIS  protocol  standard  and  a  companion  document,  commonly  known  as  the  DIS  enumera¬ 
tion  documental  specify  the  sequence  in  which  various  bits  of  information  are  transmitted  in  each  type 
of  PDU. 

Changes  in  the  PDUs'are  enabling  new  kinds  of  information  to  be  passed.  The  demonstration  of 
the  2.0  version  of  the  DIS  protocol,  for  example,  included  electromagnetic  transmissions  data,  which 
enables  the  simulation  of  radio  communication.  A  protocol  that  includes  what  is  known  strategically  and 
tactically— intelligence  about  the  enemy,  command  and  control  data,  what  is  known  about  an  enemy’s 
location— has  been  proposed.  This  would  allow  simulation  exercises  to  be  restarted  with  the  same 
amount  of  known  information  available  to  both  sides. 

The  tablemen  the  next  page  shows  some  of  the  types  of  PDUs  for  the  current  standard,  DIS  1.0, 
and  proposed  revisions  2.0  and  3.0.  The  simulation  management  protocols  proposed  for  DIS  2.0  would 
allow  new  entities  to  be  networked  without  having  to  add  code  for  modeling  them  to  each  simulator. 
They  would  also  contain  information  that  would  capture  the  state  of  the  simulation,  allowing  it  to  be  rec¬ 
reated  or  examined  at  a  later  time  DIS  3.0  PDUs  may  possibly  contain  information  for  reflecting 
changes  in  the  terrain,  such  as  when  a  bomb  has  created  a  crater  In  the  road.  New  PDUs  may  also 
provide  for  additional  capabilities  for  translating  between  models  and  simulators  with  different  levels  of 
fidelity,  representing  unaggregated,  aggregated,  and  disaggregated  entities. 


'The  latest  version  of  what  is  commonly  called  ‘Ihe  DIS  enumeration  document"  Is:  Institute  for  Simulation  and  Training,  Enu¬ 
meration  and  Bit-encoded  Values  for  Use  with  IEEE  1278. 1  Standard  for  Distributed  Interactive  Simulation— Application  Protocols, 
IST-CR-95'05  (Orlando,  FL:  Institute  for  Simulation  and  Training,  1995);  available  online  as  <URL:ftp://ftp.sc Jst.ucf.edu/public/STDS/ 
enum/enum95.doc> 

*U  S.  Army,  Simulation,  Training,  and  Instrumentation  Command,  “Distributed  Interactive  Simulation:  Operational  Concept,  “ 
draft  (Orlando,  FL  199),  p  15,  describes  types  of  PDUs  and  functions  originally  used  in  SIMNET. 

®U.S.  Army,  Simulation,  Training,  and  Instrumentation  Command,  “Battlefield  Distributed  Simulation-Developmental  Ad¬ 
vanced  Technology  Demonstration  Technology  Development  Plan,  ”  draft  (Orlando,  FL:  September  1993),  p  4. 


due  care  had  been  exercised  in  developing  a  simu¬ 
lation  plan  for  the  generation  of  simulation  data  to 
be  furnished  to  the  DAB  for  use  in  acquisition  de¬ 
cisionmaking.  It  comes  close  to  being  an  accredi¬ 
tation  procedure,  except  for  the  fact  that,  in 
accordance  with  DoD  Directive  5000.59,  only  the 
user  of  M&S  results — in  this  case,  the  DAB-can 
accredit  a  model  or  simulation.  The  EXCIMS  can 
endorse,  but  cannot  accredit,  use  of  a  model  by  the 
DAB  for  support  of  acquisition  decisionmaking. 
The  Simulation  Endorsement  Process  could  be, 


but  is  not  necessarily,  a  validation  procedure;  nor 
is  it  claimed  to  be  a  validation  procedure. 

STANDARDS  FOR  DIS 

The  simulators,  computers,  and  network  links 
used  for  DIS  would  not  work  together  unless  they 
were  all  designed  to  have  compatible  interfaces — 
i.e.,  to  comply  with  interface  standards.  Several 
types  of  standards  are  needed  for  DIS.  Industry 
standards  and  military  standards  for  computer 
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BOX  10:  Protocol  Data  Units  (PDUs)  [C(  I 

Additional  PDUs 

Additional  PDUs 

DIS  1.0  PDUs 

Proposed  for  DIS  2.0 

Considered  for  DIS  3.0 

Entity  state 

Emission 

C\ 

Fire 

Laser 

Dynamic  terrain 

Detonation 

Transmitter 

Weather/atmosphere 

Service  request 

Signal 

Fidelity  controls 

Resupply  offer 

Simulation  management 

Transfer  control 

Resupply  received 

Create  entity 

Aggregate/deaggregate 

Resupply  cancel 

Remove  entity 

Instrumentation 

Repair  complete 

Start/Resume 

Repair  response 

Stop/Freeze 

Collision 

Acknowledge 

Action  request 

Action  response 

Data  query 

Set  data 

Data 

Event 

Message 

SOURCE:  Office  of  Technology  Assessment,  1995. 

equipment  and  software  are  specified  for  DIS  ex¬ 
ercises.  However,  the  DIS  community  has  had  to 
invent  the  standards  that  are  unique  to  DIS.  These 
include  standards  for  protocols  (procedures  and 
message  formats)  for  communications  among 
simulators  and  standards  for  exercise  manage¬ 
ment. 

To  some,  the  entire  issue  of  standards  for  DIS 
revolves  around  the  creation  of  standards  for  the 
message  packets  called  protocol  data  units 
PDUs  that  are  transmitted  from  simulator  to 
simulator  over  the  network  (see  box  10).  If  these 
are  not  well  designed,  the  simulation  will  not  run 
properly,  or  will  cease  to  run  at  all.  Enunciation  of 
such  standards  was  a  primary  goal  of  the  SIMNET 
program  from  its  very  beginning.  Refinement  and 
testing  of  DIS  continues  as  a  community-wide  ac¬ 
tivity.  Industry  adoption  of  DIS  protocols  has  led 
to  the  availability  of  a  variety  of  interoperable 
simulators  (see  box  11). 

Standards  are  not  an  unqualified  good,  espe¬ 
cially  if  compliance  is  mandatory,  because  they 
are  the  result  of  compromises  among  users  that 


may  or  may  not  be  worth  making.  Standard-set¬ 
ting  usually  falls  to  an  organization  that  can  set 
and  manage  standai'ds  and  facilitate  making  simu¬ 
lations  available  to  program  managers  who  need 
them.  While  it  might  be  easy  to  get  agreement  on 
the  need  for  such  an  organization,  what  the  stan¬ 
dards  should  be  and  who  should  manage  them  is 
not  easily  agreed  on.  The  Army,  having  taken 
leadership  in  the  development  of  DIS  technology 
with  the  SIMNET  program,  called  for  a  single 
manager  that  “will  need  to  be  the  ‘policeman’  of 
the  system  with  respect  to  giving  the  equivalent  of 
the  ‘Good  Housekeeping  Seal’  to  configuration 
models  of  users’  systems  that  wish  to  interact 
throughout  the  system,  and  the  keeper  of  the  stan¬ 
dards  and  protocols  that  form  the  electronic  gate¬ 
way  into  the  electronic  battlefield’’  (170,  p.  21). 
Among  the  other  military  services,  there  was  con¬ 
cern  that  the  Army  might  be  named  the  manager 
(115).  Had  this  happened,  it  might  have  perpe¬ 
tuated  a  service  bias  to  validation  of  simulations 
and  restricted  access  to  the  DIS  platforms  on 
which  the  simulation  exercises  were  performed. 
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BOX  11;  DIS  Interoperability  Demorjstration  at  the  l/ITSE  Conference 


Industry’s  acceptance  of  the  DIS  1.0  protocols  was  demonstrated  dramatically  in  November  1992 
at  the  Interservice/industry  Training  Systems  and  Education  Conference  (l/ITSEC)  in  San  Antonio,  Tex¬ 
as.  According  to  the  DIS  Steering  Committee,  “the  I/ITSEC  is  the  training  and  simulation  community’s 
major  annual  gathering.  Part  of  the  conference  is  a  large  trade  show  to  which  all  the  major  modeling 
and  simulation  contractors  bring  their  wares.  In  the  past  all  the  simulators  demonstrated  did  so  on  a 
stand-alone  basis.  During  the  1992  show  [more  than  30]  simulators,  computer  generated  force  devices, 
and  monitoring  devices,  from  [more  than  20]  organizations,  were  linked  together  [by  Ethernet]  using  the 
basic  DIS  PDUs  The  virtual  world  consisted  of  a  military  base  near  the  Pacific  ocean  (Fort  Hunter-Lig- 
gett)  and  the  adjacent  waters.  The  scenario  included  maritime,  air-to-air,  air-to-ground,  ground-to-air, 
and  land  operations  in  which  all  the  players  took  part.  ”  This  was  the  “first  major  tesf  of  the  DIS  stan¬ 
dard  protocols  and  communication  architecture.' 

The  simulated  platforms  and  weapon  systems  included  E-2Cs,  F/A-18S,  AH-64  Apache  helicop¬ 
ters,  Ml  tanks,  a  Dragon  antitank  missile,  a  Patriot  antiaircraft  missile,  the  U.S.  S.  Ticonderoga,  the 
U.S.  S.  Perry,  \he  U.S.  S.  Wasp,  and  Soviet-made  SU-25  attack  aircraft.'Most  of  the  simulators  had  been 
designed  for  other  purposes  before  SIMNET,  and  many—including  SIMNET  simulators  that  used  the 
SIMNET  protocols— had  to  be  outfitted  with  separate  boxes  to  translate  messages  into  DIS  protocol 
data  units.  Experts  had  predicted  it  would  take  much  longer  than  two  years  to  independently  develop 
protocols  that  could  be  exchanged  across  the  network  quickly  enough  for  interaction  in  real  time." In 
this  case  the  exchange  happened  in  real  time  even  with  some  of  the  simulators  translating  protocols.  It 
was  proof  that  industry  could  fully  participate  in  the  DoD  simulation  network,  which  is  a  necessary  step 
towards  use  of  the  network  for  design,  virtual  prototyping,  and  evaluation  for  procurement. 

In  a  separate  simulation,  live  tanks  at  Fort  Hunter-Liggett  exercised  with  simulated  vehicles.  Ob¬ 
servers  at  the  l/ITSEC  could  watch  on  a  big  screen  either  a  view  of  the  terrain  televised  by  a  camera  on 
top  of  a  tank  (roughly  what  the  live  tank  crew  saw)  or  a  computer-generated  image  of  the  virtual  battle¬ 
field  from  the  tank’s  point  of  view  (roughly  what  a  tank  simulator  crew  would  see). 


’  DIS  Steering  Committee,  The  DIS  Vision,  A  Map  to  the  Future  of  Distributed  Simulation^erslon  1 ,  IST-SP-94-01  (Orlando,  FL 
The  University  of  Central  Florida,  Institute  for  Simulation  and  Training,  May  1994). 

*L.  D.  Voss,  A  Revolution  in  Simulation:  Distributed  Interaction  in  the  ‘90s  and  Beyond ArWnqXon,  VA:  Pasha  Publications,  Inc., 
1993),  p  106 

*K  Doris,  presentation,  Intersewice/industry  Training  Systems  and  Education  (l/ITSE)  Conference,  San  Antonio,  TX,  Novem¬ 
ber  1992 

SOURCE:  Office  of  Technology  Assessment,  1995 


Probably  these  concerns  have  been  alleviated  by 
the  recent  establishment  of  the  DMSO  Architec¬ 
ture  Management  Group,  which  will  specify  a 
high-level  architecture,  including  standards,  for 
DoD  distributed  simulations.  The  AMG  is  ex¬ 
pected  to  specify  the  use  of  the  DIS  protocols  de¬ 
veloped  in  the  DIS  standards  workshops  at  the 
University  of  Central  Florida’s  Institute  for  Simu¬ 
lation  and  Training  (see  box  12). 


To  standardize  the  VV&A  of  simulations, 
benchmarking  procedures  may  have  to  be  defined. 
Proposed  solutions  vary.  The  National  Security 
Industrial  Association  has  suggested  that  “we 
must  depart  from  the  concept  of  optimized  weap¬ 
on  systems  and  relative  values. . .  One  way  to  en¬ 
force  this  change  would  be  to  designate  a  scripted 
benchmark  which  any  simulation  should  be  able 
to  replicate  in  terms  of  two  forces  of  particular  di- 
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BOX  12;  DIS  Standards  Workshops 


Standardization  of  DIS  simulation  and  communications  protocols  insures  interoperability  of  all 
hardware  and  software  that  complies  with  the  standard  protocols,  DoD  decided  in  1990  to  use  industry- 
standard  simulation  protocols  if  suitable  ones  existed,  and  the  SIMNET  protocols  were  upgraded  to  a 
new  standard  known  as  the  DIS  1.0  protocols,  The  new  standard  was  been  widely  accepted  throughout 
the  simulation  industry,  as  was  demonstrated  In  1992  when  very  different  simulators  from  over  30  differ¬ 
ent  companies  were  interactively  networked  using  the  draft  DIS  1.0  protocols  in  real  time.  A  DIS  com¬ 
munity  representative  asked  the  Standards  Board  of  the  Institute  of  Electrical  and  Electronics  Engineers 
(IEEE)  to  adopt  the  DIS  1 .0  protocols  as  an  IEEE  standard,  and  it  did  so  in  1993,  The  IEEE  standard, 
called  IEEE  1278  -1993,1  was  subsequently  recognized  endorsed),  by  the  American  National  Stan¬ 
dards  Institute  (ANSI).  This  doubtless  averted  proliferation  of  competing  standards  and  concomitant 
problems  of  noninteroperability. 

The  protocols  need  to  be  revised  as  telecommunications  and  computing  technologies  advance. 
Some  of  the  protocols  used  for  DIS  are  widely  used  and  fairly  stable.  Examples  include  the  basic  wide- 
area  networking  protocols  used  by  the  Internet;  these  are  called  the  TCP/IP  protocols,  after  the  acro¬ 
nyms  for  two  of  them,  the  Transmission  Control  Protocol  and  the  Internet  Protocol.  However,  the  applica¬ 
tion-layer  protocols  used  only  for  DIS  are  revised  frequently.  New  experimental  versions  of  the  DIS  pro¬ 
tocols  are  defined  by  Government,  industry,  and  academic  participants  at  DIS  standards  workshops 
hosted  by  the  Institute  for  Simulation  and  Training  (1ST)  at  the  University  of  Central  Florida.'^This  pro¬ 
cess  allows  both  military  requirements  and  the  profit  motive  to  motivate  the  development  of  protocols 
that  are  usable  by  many  types  of  applications  and  by  systems  embodying  a  variety  of  technologies, 

Experimental  versions  of  the  DIS  protocols  are  tested  in  DoD  excercises,  and  perhaps  elsewhere, 
If  satisfactory,  a  version  may  be  proposed  for  adoption  by  the  IEEE  Standards  Board.  If  adopted,  many 
domestic  and  foreign  companies  may  produce  simulation  and  networking  hardware  and  software  that 
complies  with  the  standard,  increasing  the  options  for  nondevelopmental  procurement  of  such  products 
by  DoD.  Standards  adopted  by  the  IEEE  may  be  recognized  by  ANSI  or  adopted  by  the  International 
Electrotechnical  Committee  (lEC)  of  the  International  Organization  for  Standardization  (ISO). 

Balloting  groups  of  the  1ST  standards  working  groups  have  recently  approved  the  proposal  of  two 
draft  DIS  standards  to  the  IEEE  Standards  Board.  The  first,  P1278.1,  is  a  standard  for  protocols;  the 
second,  PI  278.2,  Is  a  standard  for  communications.  The  respective  working  groups  are  struggling  to 
resolve  most  balloting  comments  (suggestions  and  reservations)  in  time  to  submit  these  draft  standards 
to  the  IEEE  Standards  Board  in  August  for  consideration  at  its  September  1995  meeting.'lf  they  are 
adopted  this  year,  they  will  become  IEEE  standards  IEEE  1278.1-1995  and  IEEE  1278.2-1995  and  will 
complement  IEEE  1278-1993.  Balloting  for  a  draft  standard  for  exercise  management  and  feedback, 
PI  278.3,  ended  June  30,  but  tabulation  of  the  results  and  resolution  of  comments  has  not  been  com¬ 
pleted  as  this  background  paper  goes  to  press  (July  1995). 'If  it  is  adopted  in  1996,  it  will  become  IEEE 
standard  IEEE  1278.3-1996. 


'  Institute  of  Electrical  and  Electronic  Engineers,  IEEE  Standard  for  Information  Technology— Protocols  for  Distributed  Interac- 
rive  Simulation  Applications,  Entity  Information  and  Interaction,  IEEE  Standard  1278-1993  (Piscataway,  NJ  IEEE  Customer  Sen/ice 
Center,  1993). 

*  Details  are  on  the  World  Wide  Web  at  <URL.ftp://ftp.sc.ist.ucf.edu/public/STDS/wor1<shop/> 

®M.  Loper,  personal  communication,  July  17,  1995. 

SOURCE:  Office  of  Technology  Assessment,  1995 
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mensions  engaging  one  another. .  .  [ARPA’s  73 
Easting,  a  simulation  recreating  a  battle  of  the  Per¬ 
sian  Gulf  War,  is  an  example  of]  an  excellent  first 
step  in  developing  dynamic  simulation  bench¬ 
marks”  (101,  p.  39). 

Ultimately,  the  limitation  of  using  modeling 
and  simulation  throughout  the  development  life 
cycle  of  a  system  is  the  difficulty  in  relating  pro¬ 
gram  measures  of  performance  to  those  of  system 
effectiveness  and  finally  to  measures  of  force  ef¬ 
fectiveness.  Each  organization — analytic,  combat 
development,  training  and  doctrine,  test  and  eval¬ 
uation — uses  its  own  measurement  tools  and  has 
its  own  perceptions  of  how  to  measure  the  effects. 
In  order  to  pass  on  lessons  learned,  share  models 
and  modeling  tools  and  resources,  and  coherently 
track  systems  through  their  life  cycle  process, 
mutually  acceptable  tools,  common  data  dictio¬ 
naries,'^^  and  common  measures  of  effectiveness 
must  be  coordinated  (101,  p.  40). 

Without  such  benchmarking,  different  simula¬ 
tions  used  to  predict  system  effectiveness  give 
disparate  results.  One  simulation  may  indicate 
that  proposed  system  X  would  increase  the  effec¬ 
tiveness  of  an  armored  division,  in  a  particular 
scenario,  by  half,  while  another  simulation  may 
find  no  significant  effect.  Such  occurrences  some¬ 
times  lead  to  allegations  of  biases  in  “legacy” 
(long-used,  service-sponsored)  simulations. 
However,  not  all  differences  in  preference  deserve 
the  peijorative  term  “bias.”  Differing  roles  and 
missions  can  lead  to  some  very  defensible  differ¬ 
ences  in  preference.  The  test  and  acquisitions 
communities  illustrate  this  point:  the  approaches 
of  the  two  communities  are  in  direct  opposition  to 
each  other.  The  acquisition  community  uses  rela¬ 
tive  changes  in  outcome  as  their  primary  measure 
of  effectiveness  (MOE).  Scenarios  are  normalized 
with  optimal  operation  assumed  for  all  weapons  to 
better  detect  inherent  differences  when  comparing 


different  systems,  models,  or  designs.  These  sim¬ 
ulations  are  not  necessarily  intended  to  reflect 
reality.  The  testing  and  training  communities, 
however,  seek  a  realistic  representation  of  the  fog 
and  friction  of  war  (101,  p.  3).  These  differences 
stem  from  the  communities’  differing  missions: 
the  acquisition  community  seeks  the  best  item,  or 
the  best  item  for  the  money,  while  the  testing  com¬ 
munity  seeks  the  Achilles  heel  of  a  given  item. 

SCALABILITY  AND  SCALING 

The  draft  DoD  Modeling  and  Simulation  Master 
Plan  (195)  defines  scalability  as  “the  ability  of  a 
distributed  simulation  to  maintain  time  [sic]  and 
spatial  consistency  as  the  number  of  entities  and 
accompanying  interactions  increase”  (cf.  (27))  It 
is  desirable  that  DoD  distributed  simulations  be 
scalable,  e.g.,  in  order  to  provide  large  forces  for 
high-echelon  commanders  to  command.  To  this 
end,  ARPA  at  one  time  wanted  100,000  entities  to 
be  networked  in  its  Synthetic  Theater  of  War  pro¬ 
gram  (162,164).  It  is  also  desirable  that  distrib¬ 
uted  engineering  simulations,  such  as  those  being 
developed  for  ARPA’s  Simulation-Based  Design 
program,  be  scalable  to  encompass,  e.g.,  simula¬ 
tion  of  all  systems  on  a  ship  (147). 

However,  there  is  a  limit  to  the  number  of  enti¬ 
ties  that  can  participate  in  a  distributed  simulation 
without  causing  loss  of  temporal  consistency — 
i.e.,  desynchronization.'^^  The  limit  is  not  a 
constant  of  nature,  like  the  speed  of  light;  it  may 
be  raised  by  improving  hardware,  software,  com¬ 
munications  architecture,  and  exercise  manage¬ 
ment  (7,39,  81,102, 125, 130, 132).  However,  it  is 
a  real  limitation,  one  that  is  recognized  by  the  draft 
DoD  Modeling  and  Simulation  Master  Plan. 

Limits  to  scalability  were  encountered  in  the 
STOW-E  exercise,  even  though  STOW-E  set  a 
new  record  for  DIS  scale  (over  2,000  simulta- 


48  Data  dictionaries  are  documents  that  define  and  identity  needed  data.  An  entry  in  a  data  dictionary  .specifies  what  the  name  of  a  data  set 
means,  what  the  data  are  used  for,  where  the  data  are  stored,  how  the  data  are  transferred  and  used,  and  the  form  tliey  are  put  in. 

There  is  also  a  limit  to  the  niunber  of  types  of  entities,  although  this  is  a  limit  imposed  more  by  the  affordability  of  pre-exercise  simulation 
development  than  by  run-time  constraints.  See  reference  (117), 
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neous  entities)  by  demonstrating  several  effective 
communications  techniques  that  had  been  pro¬ 
posed  (27)  to  widen  PDU  traffic  bottlenecks;  these 
included  bundling, compression,^^  multicast- 
ing,^2  and  the  use  of  “delta-PDUs”  (PDUs  that 
announce  changes  rather  than  status).  One  analyst 
of  STOW-E  data  wrote  that  “in  some  machines, 
computer  processing  was  overloaded  after  entity 
counts  reached  1,000,  with  others  having  a  slight¬ 
ly  higher  threshold”  (119). 

OTA  staff  observing  STOW-E  at  ARPA’s  WIS- 
SARD  (What  If  Simulation  System  for  Advanced 
Research  and  Development)  facility  and  other 
sites  noted  that  some  briefers  described  the  STOW 
program  as  having  a  goal  of  netting  50,000  enti¬ 
ties  in  its  STOW-97  demonstration  (a  change  indi¬ 
cating  greater  concern  about  scalability),  while 
other  briefers  still  quoted  the  original  goal  of  net¬ 
ting  100,000  entities.  ARPA’s  World  Wide  Web 
page  describing  STOW  efforts  still  says  the  goal  is 
netting  100,000  entities  (164). 

Even  if  it  were  technically  feasible  to  have 
100,000  tank  simulators  drive  and  fight  on  the 
same  virtual  battlefield,  they  would  not  represent 
real  divisions  or  corps  (or  higher  echelon  orga¬ 
nizations)  unless  each  small  unit’s  “slice”  (share) 
of  division-  or  corps-level  intelligence,  coimnu- 
nications,  logistics,  and  other  support  functions  is 
modeled.  That  is,  balanced  representation  of  dif¬ 
ferent  types  of  entities  is  necessary  for  fidelity. 

Attempts  to  increase  fidelity  by  simulating 
more  details— e.g.,  of  radio  signals,  or  a  dis¬ 
mounted  soldier’s  pose — ^will  also  stress  scalabil¬ 


ity.  Other  things  being  equal,  simulating  more 
detail  will  require  simulating  fewer  entities. 

FUNDING  FOR  DIS  SYSTEMS 

It  is  difficult  to  estimate  how  much  DoD  spends 
on  combat  modeling  and  simulation,  partly  be¬ 
cause  there  are  disputes  about  how  much  of  the 
Department’s  various  activities  are  modeling  and 
simulation,  and  partly  because  much  modeling 
and  simulation — including  combat  modeling  and 
simulation — ^is  performed  as  an  integral  part  of  ac¬ 
quisition,  operations,  and  maintenance  and  is  not 
funded  as  a  sepai'ate  account.  This  section  reports 
some  estimates  of  funding  for  modeling  and  simu¬ 
lation,  especially  for  DIS. 

I  Research  and  Development 

The  Modeling  and  Simulation  Technology  Area 
Assessment  (TAA)  prepared  by  the  Office  of  the 
Director  of  Defense  Research  and  Engineering  in 
1994  (186)  tallied  and  proposed  DoD  investment 
in  development  of  models  and  simulations.  Devel¬ 
opment  of  the  1994  M&S  Technology  Area  Plan 
(TAP),  conduct  of  the  1994  M&S  Technology 
Area  Review  (TAR),  and  production  of  the  1994 
M&S  TAA  were  DoD’s  first  attempt  to  compre¬ 
hensively  analyze  and  plan  the  course  of  develop¬ 
ment  of  M&S  technology.  In  the  process,  DoD 
discovered  that  the  Army  and  ARPA  treated  M&S 
as  a  unique  technology  area;  while  the  Air  Force, 
the  Navy,  and  the  Ballistic  Missile  Defense  Orga¬ 
nization  (BMDO)  considered  M&S  development 


Bundling  is  the  concatenation  of  the  information  fields  of  several  datagrams  addressed  to  the  same  destination  into  one  longer  infonna- 
don  field,  which  requires  only  one  header  and  set  of  flags.  This  reduces  network  congestion  but  requires  delaying  transmission  of  the  resulting 
packet  until  its  information  field  is  filled  with  the  component  infonnation  fields, 

^  ^  Compression  is  the  coding  of  information  in  a  datagram  so  as  to  reduce  the  number  of  characters  required .  The  basic  principle  is  to  use  the 
shortest  code  words  for  tlie  character  strings  that  occur  most  frequently  in  tlie  uncompressed  data.  In  DIS  exercises  such  as  STOW-E,  compres¬ 
sion  has  reduced  packet  size  to  4  or  5  percent  of  the  uncompressed  size,  in  effect  increasing  the  number  of  simulators  tliat  can  participate  by  a 
factor  of  20  or  25. 

^^Multicasting  is  the  addressing  of  packets  to  an  interest  group  rather  Uian  to  a  particular  computer,  or  as  had  been  done  in  SIMNET,  to  all 
computers.  Thus  a  gateway  between  a  WAN  (such  as  the  Defense  Simulation  Internet)  and  a  LAN  connecting  aircraft  simulators  can  be  pro¬ 
grammed  to  ignore  packets  arriving  from  tlie  WAN  that  are  addressed  to  an  acoustic  emissions  interest  group;  tliat  is,  tlie  gateway  will  not  pass 
them  on  to  the  LAN,  which  serves  no  simulators  tliat  need  to  receive  tliem.  This  reduces  congestion  of  the  LAN. 


54  Distributed  Interactive  Simulation  of  Combat 


TABLE  b:  Funding  for  Research  and  Development  of  Modeling  and  Simulation, 
by  Technology  Sub-area 

($  millions) 

subarea 

FY1994 

FY  1995 

FY  1996 

Architecture  &  standards 

$114.8 

$88.2 

$83.4 

Environmental  representation 

68.4 

76,4 

64.9 

Computer  generated  forces 

44.4 

54.4 

49.9 

Total 

$227.6 

$219.0 

$198,2 

SOURCE"  Department  of  Defense,  Defense  Modeling  and  Simulation  Office,  1994 


as  supporting  technology  (tools)  for  other  pro¬ 
grams.  The  Joint  Directors  of  Laboratories  (JDL) 
had  not  kept  track  of  laboratory  spending  on  mod¬ 
eling  and  simulation  as  a  separate  account.  The 
TAA  concluded  that,  because  of  the  “confusion” 
then  existing  among  the  DoD  Components  over 
the  distinction  between  M&S  technology  devel¬ 
opment  and  M&S  applications,  the  first  TAA  is 
probably  incomplete  and  “significant  6.2-6.3 
funded  M&S  technology  development  work,  be¬ 
yond  that  reported  in  the  TAR,  is  being  conducted 
in  support  of  other  technology  areas,  as  well  as  in 
6.4  funded  Demonstration  and  Validation  pro¬ 
grams”  (186).  Nevertheless,  the  TAA  concluded 
that  preparation  of  the  first  TAP,  TAR,  and  TAA 
was  very  productive  for  DoD’s  M&S  community. 
It  identified  many  excellent  programs  as  well  as 
areas  needing  increased  emphasis.  “The  TAR,  in 
particular,  offered  an  excellent  forum  for  cross - 
fertilization  among  components  and  among  pro¬ 
grams  within  components.  The  lessons  learned 
from  this  initial  effort  will  yield  significant  im¬ 
provements  in  subsequent  TAPs,  TARs,  and 
TAAs  ”  (186).  The  1995  TAP  is  expected  to  be  re¬ 
leased  shortly  after  this  background  paper  is  com¬ 
pleted. 

Despite  its  warning  of  incompleteness,  the 
1994  TAA  tallied  M&S  Technology  Area  funding 
of  approximately  $232  million  in  FY  1994,  $231 
million  in  FY  1995,  and  $226  million  in  FY  1996, 
before  adjustments  were  made  by  and  overhead 
expenses  were  deducted  for  the  Office  of  the  Sec¬ 
retary  of  Defense.  The  TAA  identified  the  funding 
of  several  projects  in  nine  program  elements.  The 
TAA  panel  binned  the  funding  into  three  “sub¬ 


areas”  of  technology  flagged  as  meriting  special 
emphasis:  Architecture  and  Standards,  Environ¬ 
mental  Representation,  and  Computer  Generated 
Forces.  (See  table  5.)  These  subareas  are  critical  to 
the  development  of  distributed  simulations,  in¬ 
cluding  DIS. 

The  Defense  Modeling  and  Simulation  Office’s 
own  1995  Modeling  and  Simulation  Investment 
Plan  projects  a  doubling  of  funding,  from  about  $5 
million  in  FY  1995  to  about  $10  million  in  FY 
1996  (67,  slide  38).  (See  table  6.)  All  of  the  activi¬ 
ties  listed  support  DIS;  one — the  DIS  enumera¬ 
tion  document — is  exclusively  for  DIS. 

Another  DIS-related  project  is  the  Universal 
Threat  System  for  Simulators  (UTSS),  which  will 
build  a  data  repository  for  threat  data  required  by 
the  supported  simulators.  It  will  also  build  a  Real- 
Time  Simulation  Software  (RTSS)  library  that 
will  host  reusable,  validated  simulation  software 
and  will  catalog  available  threat  simulations 
(144).  The  initial  data  modeling  effort,  now  com¬ 
plete,  was  gathering  detailed  data  requirements 
for  each  of  the  four  simulators  selected  for  inclu¬ 
sion  in  the  initial  development  of  UTSS:  the 
Army’s  Modular  Semiautomated  Forces  (Mod- 
SAF),  the  Navy’s  F-14D  Weapon  Systems  Train¬ 
er,  the  Marine  Corps’s  UH-IN  Weapon  Systems 
Trainer,  and  the  Air  Force’s  F-15C  and  F-16C 
UTDs.  The  project,  formerly  a  program,  received 
funding  from  DMSO  in  FY  1993  and  FY  1994.  It 
is  now  funded  solely  by  the  Navy,  as  project 
W2124  of  program  element  0204571  N,  Consoli¬ 
dated  Training  Systems  Development  (122),  for 
which  DoD  budgeted  $48  million  in  FY  1996  and 
$38  million  in  FY  1997  (190). 
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TABLE  6;  The  Defense  Modeling  and  Simulation  Office's  Investment  Plan 

($  thousands) 

FY  1995 

Estimated 

FY1996 

Functional  Data  Administrator  (FDAd)‘ support 

$1,184,5 

$2,900.0 

Model  and  Simulation  Resource  Repository  (MSRR)  projedt 

1,443.8 

5,700.0 

Data  verification,  validation,  and  certification  tasks 

680.0 

1,000.0 

Authoritative  data  sources  (ADS),  portabie  data  quality  tool,' 
and  C*  core  data  model  integration 

300.0 

— 

Modeling  and  simulation  community  directories 

411.0 

— 

Data  security 

575.0 

1,000.0 

DIS  enumeration  document 

533,0 

— 

Total 

$5,127.3 

$10,600.0 

‘The  Defense  Modeling  and  Simulation  Office  (DMSO)  has  been  designated  the  Modeling  and  Simulation  Functional  Data  Administrator  (M&S 
FDAd). 

’DMSO  has  established  a  MSRR  on  the  World  Wide  Web  at  Uniform  Resource  Locator  (URL)  <URL:http://www .dmso.mil/dmso/msrr.html>  as  part 
of  the  DoD  Data  Repository  System.  It  has  hypertext  links  to  other  MSRR  nodes,  including  the  DMSO  Interim  MSRR  hosted  on  the  Joint  Data  Base 
Elements  for  Modeling  and  Simulation  node  at  the  U.S.  Army  Electronic  Proving  Ground  at  Fort  Huachuca,  Arizona,  at  <URL:http://huachuca- 
idbe.army.mil/>. 

‘The  supported  activity  is  making  computer  software  used  for  data  verification  at  U.S.  Central  Command  portable,  I.e. ,  capable  of  being  used  on  a 
variety  of  computers 

"Modeling  and  simulation  communrty  directories  enable  M&S  developers  and  users  to  locate  authoritative  data  sources,  databases,  models  and 
simulations, 

•The  latest  version  of  what  is  commonly  called  the  DIS  enumeration  document  Is:  Institute  for  Simulation  and  Training,  Enumeration  and  Bitten’ 
coded  Values  for  Use  with  IEEE  1278.1  Standard  for  Distributed  Interactive  Simulatioir— Application  Protocols,  IST-CR-95-05  (Orlando,  FL:  Insti¬ 
tute  for  Simulation  and  Training,  1995).  Available  online  at  <URL:ftp://ftp.sc.ist.ucf.edu/public/STDS/enum/enum95.doc>,  flledate  Mar  21 , 1995. 
SOURCE:  Department  of  Defense,  1995. 


,  Procurement 

DoD  has  not  tallied  funding  for  procurement  of 
modeling  and  simulation  equipment  and  services 
in  general  or  for  DIS  in  particular.  In  the  past,  no 
need  for  such  an  accounting  was  recognized,  and 
doing  it  would  be  problematic  because  of  defini¬ 
tional  difficulties.  An  expansive  definition  of  sim¬ 
ulation  says  that  “everything  is  simulation  except 
combat”  (191 ).  According  to  this  definition,  most 
of  the  defense  budget  is  for  simulation,  but  less  ex¬ 
pansive  defininitions  of  simulation  would  count 
only  a  small  fraction  of  the  defense  budget  as 
funding  for  simulation,  and  only  a  fraction  of  that 
would  be  funding  for  DIS.  However,  as  the  Army 
proceeds  with  its  program  of  “digitizing  the 
battlefield”  and  as  DoD,  guided  by  the  MSMP, 
proceeds  to  interface  live  forces,  constructive  sim¬ 
ulations,  and  distributed  simulations  to  one  anoth¬ 
er  and  to  the  Global  Command  and  Control 
System — so  that  forces  can  train  at  home  and 


when  they  deploy — it  will  become  more  difficult 
to  identify  systems  that  are  not  from  time  to  time 
involved  in  DIS. 

Thus  again  at  the  risk  of  undercounting,  we  can 
mention  some  of  the  lai-gest  DIS-related  programs 
in  the  DoD  Comptroller’s  1995  Procurement  Pro¬ 
grams  (P-1)  report.  Top  ranked  (in  terms  of  FY 
1997  funding)  is  SIMNET/Close  Combat  Tactical 
Trainer,  for  which  $88.5  million  is  requested.  The 
Close  Combat  Tactical  Trainer  (CCTT),  a  descen¬ 
dant  of  SIMNET,  is  the  first  of  several  planned 
Army  DIS  programs  that  are  collectively  called 
the  Combined  Arms  Tactical  Trainer  (CATT)  pro¬ 
gram.  Its  aim  is  to  network  simulators  for  a  variety 
of  Army  combat  platforms  for  ±e  purpose  of  col¬ 
lective  combined  arms  tactical  training.  The  se¬ 
cond  CATT  program,  which  ranks  second  among 
DIS  programs  in  FY  1997  P-1  funding,  is  the  Fire 
Support  Combined  Arms  Tactical  Trainer 
(FSCATT),  for  which  the  Army  has  requested  no 
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procurement  funds  through  FY  1996;  but  the 
Army  has  requested  $22  million  in  FY  1997.” 
Next  is  the  Reconfigurable  Simulators  program 
(see  figure  1-8),  for  which  the  Army  has  requested 
about  $17  million  in  FY  1997.  Some  of  the  “simu¬ 
lators,  all  types,”  for  which  the  Army  requested 
about  $8  million  in  FY  1997,  might  be  used  for 
DIS  on  occasion. 

I  Operations 

Defense  authorization  and  appropriation  bills  typ¬ 
ically  provide  little  specific  direction  on  funding 
for  modeling  and  simulation.*^ 

The  National  Defense  Authorization  Act  for 
Fiscal  Year  1995  (Public  Law  103-337)  autho¬ 
rized  $89.1  billion,  and  the  Department  of  De¬ 
fense  Appropriations  Act,  1995  (Public  Law 
103-335)  appropriated  $80.9  billion,  for  opera¬ 
tions  and  maintenance,  including  training.  The 
fraction  spent  on  DIS  activities  is  unknown  but 
presumably  very  small. 

DARPA’s  SIMNET  program  ended  in  1989, 
but  the  simulators,  communications  protocols, 
and  network  developed  by  the  program  are  still 
being  used  and  upgraded  by  ARPA’s  Advanced 
Distributed  Simulation  (ADS)  program,  the 
Army’s  Battlefield  Distributed  Simulation-De¬ 
velopmental  (BDS-D)  program  and  other  Depart¬ 
ment  of  Defense  (DoD)  programs.  The  name 
SIMNET  still  occurs  in  the  “SIMNET/Close 
Combat  Tactical  Trainer”  line  item  in  the  DoD 
Comptroller’s  1995  Procurement  Programs  (P-1) 
report  to  Congress  (189).  The  term  is  still  applied 
to  simulators,  networks,  and  simulations  that  still 
use  the  SIMNET  communications  protocols, 
which  have  been  superseded  by  the  Distributed  In¬ 
teractive  Simulation  (DIS)  protocols. 

A  distributed  interactive  simulation  is  “distrib¬ 
uted”  in  the  sense  that  simulator  computers  at 


SOURCE:  U.S.  Army  Simulation,  Training,  and  Instrumentation  Com¬ 
mand. 


multiple  training  sites  nationwide  are  connected 
by  means  of  a  local-area  network  (LAN;  see  figure 
2),  which  in  turn  maybe  connected  to  a  wide-area 
network  (WAN)  such  as  the  the  Defense  Simula¬ 
tion  Internet  (DSI)  (see  figure  3).  Trainees  can  en¬ 
ter  disparate  tank,  helicopter,  or  other  simulators 
and  all  participate  in  the  same  simulated  combat. 
The  simulation  is  “interactive”  in  the  sense  that 
humans  interact  with  the  computers  to  influence 
the  simulation.  The  battle  does  not  follow  a  set 
script,  and  trainees  win  or  lose  on  the  basis  of  their 
performance.  The  trainees  fight  a  similarly  net¬ 
worked  opposing  force,  which  may  be  a  com¬ 
bination  of  virtual  forces-real  people  operat¬ 
ing  simulators-an  d  semiautomated  forces 
(SAF)— vehicles  and  weapons  simulated  by  com¬ 
puters  with  humans  providing  operational  super¬ 
vision.  In  both  the  virtual  forces  and  the 
semiautomated  forces,  human  behavior  is  simu¬ 
lated  by  humans,  albeit  humans  not  affected  by  the 
many  stresses  of  real  combat. 


’’FSCATT  is  one  of  five  acquisition  programs  that  the  Federal  Acquisition  Streamlining  Act  of  1994(Public  Law  103-355)  authorized  the 
Secretary  of  Defense  to  designate  as  a  “pilot”  program  to  deteimine  “die  potential  for  increasing  tlie  efficiency  and  effectiveness  of  die  ac¬ 
quisition  process  using  standard  commercial  practices”  (148,  p.  65). 

^'The  National  Defense  Audiorization  Act  for  Fiscal  Year  1994  (Public  Law  103-160)  did  authorize  appropriation  of  $75  million  for  pro¬ 
curement  of  simulation  equipment  for  reserve  components. 
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21CLW 

21st  Century  Land  Warrior 

A2ATD 

Anti-Armor  Advanced 
Technology  Demonstration 

AAR 

after-action  review 

ACT 

Advanced  Concepts  and 
Technology 

ADCATT 

Air  Defense  Combined  Arms 
Tactical  Trainer 

ADS 

Advanced  Distributed 
Simulation 

ADS 

Authoritative  Data  Sources 

ADST 

Advanced  Distributed 
Simulation  Technology 

AI 

artificial  intelligence 

ALSA 

Air-Land-Sea  Agency 

ALSP 

Aggregate  Level  Simulation 
Protocol 

AMC 

Army  Material  Command 

AMG 

Architecture  Management 
Group 

ANSI 

American  National  Standards 
Institute 

ARPA 

Advanced  Research  Projects 
Agency  (formerly  DARPA) 

ARTEP 

Army  Test  and  Evaluation 
Program 

AS  ARC 

Army  Systems  Acquisition 
Requirements  Council 

ASPT 

Advanced  Simulator  for  Pilot 
Training 

ASTO 

[ARPA]  Advanced  Systems 
Technology  Office 

ATACC 

Advanced  Tactical  Air 
Command  Center 

ATCCS 

Army’s  Tactical  Command 
and  Control  System 

ATD 

Advanced  Technology 
Demonstration 

ATD 

aircrew  training  device 

ATM 

Aircrew  Training  Manual 

ATM 

anti-tank  missile 

ATM 

asynchronous  transfer  mode 

BAA 

Broad  Agency  Announcement 

BBN 

Bolt,  Beranek,  and  Newman 

BBS 

Brigade/Battalion  Battle 
Simulation 

BDS-D 

Battlefield  Distributed 
Simulation-Developmental 

BEA 

Bureau  of  Economic  Analysis 

BFTT 

Battle  Force  Tactical  Training 

BOC 

Bureau  of  the  Census 

C2 

command  and  control 

C2 

command  and  control 

C2T2 

Commercial  Communications 
Technology  Testbed 
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q2j2 

Commercial  Communications 
Technology  Testbed 

C4I 

Command,  Control, 
Communications,  Computers, 
and  Intelligence 

C^I 

Command,  Control, 
Communications,  Computers, 
and  Intelligence 

C4IFTW 

C4I  for  the  Warrior 

CAAN 

Combined  Arms  Assessment 
Network 

CASTFOREM 

Combat  Arms  Task  Force 
Engagement  Model 

CATT 

Combined  Arms  Tactical 
Trainer 

CAX 

computer-assisted  exercise 

CBD 

Commerce  Business  Daily 

CBS 

Corps  Battle  Simulation 

CCD 

charge-coupled  devices 

CCTT 

Close  Combat  Tactical 

Trainer 

CEM 

Concepts  Evaluation  Model 

CFOR 

Command  Forces 

CGF 

computer-generated  forces 

CGI 

computer-generated  imagery 

CGI 

Common  Gateway  Interface 

CIG 

computer  image  generation 

CIG 

computer  image  generator 

CINC 

Commander  in  Chief 

CJCS 

Chairman  of  the  Joint  Chiefs 
of  Staff 

CJCSI 

Chairman  of  the  Joint  Chiefs 
of  Staff  Instruction 

CL 

constructive/live 

CMTC 

Combat  Maneuver  Training 
Center 

COEA 

cost  and  operational 
effectiveness  analysis 

COFT 

Conduct  of  Fire  Trainer 

COMPASS 

Common  Operation  Mission 
Planning  and  Support 

Strategy 

Costar 

Corrective  Optics  Space 
Telescope  Axial  Replacement 

COTR 

Contracting  Officer  Technical 
Representative 

COTS 

commercial  off-the-shelf 

CP 

command  post 

CPX 

command  post  exercise 

CRADA 

Cooperative  Research  and 
Development  Agreement 

CRT 

cathode  ray  tube 

CV 

constructive/virtual 

CVL 

constructive/virtual/live 

DA 

Department  of  the  Army 

DAB 

Defense  Acquisitions  Board 

DARPA 

Defense  Advanced  Research 
Projects  Agency  (now  ARPA) 

DCSOPS 

Deputy  Chief  of  Staff  of  the 
Army  for  Operations 

DDR&E 

Director  of  Defense  Research 
and  Engineering 

DDS 

Distributed  Defense 
Simulation 

DEEM 

Dynamic  Environmental 
Effects  Model 

DI 

Demonstration  Integration 

DIA 

Defense  Intelligence  Agency 

DII 

Defense  Information 
Infrastructure 

DIS 

Distributed  Interactive 
Simulation 

DISA 

Defense  Information  Systems 
Agency 

DISN 

Defense  Information  System 
Network 

DIVE 

Dismounted  Infantry  Virtual 
Environment 

DMA 

Defense  Mapping  Agency 

DMSO 

Defense  Modeling  and 
Simulation  Office 

DoD 

Department  of  Defense 

DR&E 

Defense  Research  and 
Engineering 

DSCS 

Defense  Satellite 
Conununication  System 

DSI 

Defense  Simulation  Internet 

DSP 

Defense  Support  Program 

DT&E 

developmental  test  and 
evaluation 

DTIC 

Defense  Technical 
Information  Center 
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E2DIS 

Environmental  Effects  for 
Distributed  Interactive 
Simulations 

EMD 

engineering,  manufacturing, 
and  development 

ENCATT 

Engineering  Combined  Arms 
Tactical  Trainer 

ERINT 

Extended  Range  Interceptor 

EW 

electronic  warfare 

EXCIMS 

Executive  Council  for 
Modeling  and  Simulation 

FAA 

Federal  Aviation 
Administration 

FFRDC 

federally  funded  research  and 
development  center 

FLIR 

forward-looking  infrared 

FSCATT 

Combined  Arms  Tactical 
Trainer 

FWG 

functional  working  group 

GCCS 

Global  Command  and  Control 
System 

GOSIP 

Government  Open  Systems 
Interconnection  Protocol 

GOTS 

government  off-the-shelf 

GPS 

Global  Positioning  System 

HMD 

head-mounted  display 

HOTAS 

hands  on  throttle  and  stick 

l4  WISSARD 

Intelligent,  Imaginative, 
Innovative,  Interactive  What 

If  Simulation  System  for 
Advanced  Research  and 
Development 

I&P 

intelligence  and  planning 

lAC 

Information  Analysis  Center 

IADS 

Integrated  Air  Defense 
Simulator 

IDA 

Institute  for  Defense  Analyses 

IEEE 

Institute  of  Electrical  and 
Electronic  Engineers,  Inc. 

lEI 

Illusion  Engineering,  Inc. 

IFF 

identification  friend/foe 

IFOR 

Intelligent  Forces 

I/ITSE 

Interservice/Industry  Training 
Systems  and  Education 

iMSRR 

Interim  Modeling  and 
Simulation  Resource 
Repository 

IP 

Internet  Protocol 

IPPD 

integrated  product  and 
process  development 

IRIAM 

Integrated  Radar/Infrared 
Analysis  and  Modeling 

System 

1ST 

Institute  for  Simulation  and 
Training 

lUSS 

Integrated  Unit  Simulation 
System 

IVIS 

Inter- Vehicular  Information 
System 

IW 

information  warfare 

J-6 

[Joint  Staff]  Command, 
Control,  Communications, 
and  Computing  Systems 
Directorate 

J-7 

[Joint  Staff]  Operational  Plans 
and  Interoperability 

Directorate 

J-8 

[Joint  Staff]  Force  Structure, 
Resources,  and  Assessment 
Directorate 

JAST 

Joint  Advanced  Strike 
Technology  [Program] 

JCS 

Joint  Chiefs  of  Staff 

JDC 

Joint  Doctrine  Center 

JDL 

Joint  Directors  of 

Laboratories 

JECEWSI 

Joint  Electronic  Combat 
Electronic  Warfare  Simulation 

JICM 

Joint  Integrated  Contingency 
Model 

JM&S 

Joint  Models  and  Simulations 

JMASS 

Joint  Modeling  and 

Simulation  System 

JMCIS 

Joint  Maritime  Command 
Information  System 

J-METL 

Joint  Mission-Essential  Task 
List 

JPO 

Joint  Program  Office 
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JRTC 

Joint  Readiness  Training 
Center 

JSEP 

Joint  Simulation  Endorsement 
Process 

JSIMS 

Joint  Simulation  System 

JSTARS 

Joint  Surveillance  Target 
Attack  Radar  System 

JTCTS 

Joint  Tactical  Combat 

Training  System 

JTN 

Joint  Targeting  Network 

JWARS 

Joint  Warfare  Simulation 

JWC 

Joint  Warfare  Center 

JWFC 

Joint  Warfighting  Center 

JWID 

Joint  Warrior  Interoperability 
Demonstration 

LAM 

Louisiana  Maneuvers 

LAN 

local  area  network 

LCLV 

liquid  crystal  light  valve 

LHN 

long-haul  network 

LOSAT 

Line-of-Sight  Anti-Tank 

M&S 

modeling  and  simulation 

MARS 

Multiwarfare  Assessment  and 
Research  System 

MDT2 

Multiservice  Distributed 
Training  Testbed 

MEL 

Master  Environmental 

Library 

METL 

Mission  Essential  Task  List 

MILES 

Multiple  Integrated  Laser 
Engagement  System 

MLS 

multilevel  security 

ModSAF 

Modular  Semiautomated 
Forces 

MORS 

Military  Operations  Research 
Society 

MOSAIC 

Models  &  Simulations:  Army 
Integrated  Catalog 

MSE 

Mobile  Subscriber  Equipment 

MSIS 

Modeling  and  Simulation 
Information  System 

MSMP 

Modeling  and  Simulation 
Master  Plan 

MSRR 

Modeling  and  Simulation 
Resource  Repository 

M-STOW 

Maritime  Synthetic  Theater  of 
War 

MSWG 

Modeling  and  Simulation 
Working  Group 

NMS 

National  Military  Strategy 

NRL 

National  Research  Laboratory 

NTC 

National  Training  Center 

NTIS 

National  Technical 

Information  Service 

NTSA 

National  Training  Systems 
Association 

NTSC 

Naval  Training  Systems 

Center 

OASD  (FM&P) 

Office  of  the  Assistant 
Secretary  of  Defense  (Force 
Management  and  Personnel) 

OPFOR 

opposing  force 

OR 

operational  requirements 

ORSBM 

Oak  Ridge  Spreadsheet  Battle 
Model 

ORTA 

Office  of  Research  and 
Technology  Applications 

OSD 

Office  of  the  Secretary  of 
Defense 

OSI 

Open  Systems  Interconnect 

OSTP 

Office  of  Science  and 
Technology  Policy 

OT&E 

operational  test  and 
evaluation 

PDU 

protocol  data  unit 

PE 

program  element 

PEO 

program  executive  office(r) 

POTS 

Precision  Gunnery  Training 
System 

PM 

program  manager 

PM 

project  manager 

PM  CATT 

Project  Manager  for  the 
Combined  Arms  Tactical 
Trainer 

PMDIS 

Project  Manager  for 
Distributed  Interactive 
Simulation 

PM  TRADE 

Project  Manager  for  Training 
Deviees 

PO 

project  office 

Preview 

Prevent  Real  Errors  by  Virtual 
Investigation  of  Engineering 
Worlds 
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R&D 

research  and  development 

RD&A 

Research,  development,  and 
Acquisition 

RDT&E 

research,  development,  test, 
and  evaluation 

REA 

Remote  Entity 

Approximation 

REFORGER 

Return  of  Forces  to  Germany 

RF 

radio-frequency 

RFPI 

Rapid  Force  Projection 
Initiative 

RTSS 

Real-Time  Simulation 
Software 

S&T 

science  and  technology 

SAF 

semiautomated  forces 

SAFOR 

semiautomated  forces 

SAIC 

Science  Applications 
International  Corp. 

SBIR 

Small  Business  Innovation 
Research  program 

SDIO 

Strategic  Defense  Initiative 
Office 

SDIS 

Simulation  Data  Interchange 
Specification 

SEES 

Security  Exercise  Evaluation 
System 

SEI 

Software  Engineering 

Institute 

SEI 

System  Engineering  and 
Integration 

SEM 

Simulation  Engineering  and 
Modeling 

SIF 

SSDB  [Standard  Simulator 
Data  Base]  Interchange 
Format 

SIF/DP 

SSDB  Interchange  Format  for 
Distributed  Processing 

SIF/HDI 

SSDB  Interchange  Format  for 
High  Detail  Input/Output 

SIH 

Soldier  Information  Highway 

SIMITAR 

Simulation  In  Training  for 
Advanced  Readiness 

SIMNET 

Simulator  Network 

SINCGARS 

Single  Channel  Ground  and 
Airborne  Radio  System 

SIPE 

Soldier  Integrated  Protective 
Ensemble 

SMART 

Susceptibility  Model 
Assessment  and  Range  Test 

SOFNET 

Special  Operational  Forces 
Inter  Simulator  Network 

SSDB 

Standard  Simulator  Data  Base 

STOW 

Synthetic  Theater  of  War 

STRICOM 

[U.S.  Army]  Simulation, 
Training,  and  Instrumentation 
Command 

T&E 

test  and  evaluation 

TAA 

Technology  Area  Assessment 

TACS/TADS 

Tactical  Air  Control 
System/Tactical  Air  Defense 
System 

TAP 

Technology  Area  Plan 

TAR 

Technology  Area  Review 

TASC 

The  Analytic  Sciences 
Corporation 

TCP 

Transmission  Control 

Protocol 

TCP/IP 

Transmission  Control 
Pi'otocol/Internet  Protocol 

TGT 

Tank  Gunnery  Trainer 

THAAD 

Theater  High  Altitude  Area 
Defense 

TOW 

tube-launched,  optically 
tracked,  wire-command  link 
guided  [missile] 

TQM 

total  quality  management 

TRADOC 

Training  and  Doctrine 
Command 

TSI 

Technology  Systems,  Inc. 

TTGT 

Tank  Team  Gunnery  Trainer 

TTTS 

Tanker-Transport  Training 
System 

TV 

television 

TWBnet 

Terrestrial  Wideband  Network 

TWG 

technology  working  group 

TWSTIAC 

Tactical  Warfare  Simulation 
and  Technology  Information 
Analysis  Center 

UCOFT 

Unit  Conduct  of  Fire  Trainer 

UJTL 

Universal  Joint  Task  List 
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URL 

Uniform  Resource  Locator 

System 

USACOM 

U.S.  Atlantic  Command 

V&V 

verification  and  validation 

USA  STRICOM 

U.S.  Army  Simulation, 

VL 

virtual/live 

Training,  and  Instrumentation 

VV&A 

verification,  validation,  and 

Command 

accreditation 

USD(A&T) 

Undersecretary  of  Defense  for 

VV&C 

verification,  validation,  and 

Acquisition  and  Technology 

certification 

USD(P&R) 

Undersecretary  of  Defense  for 

WAN 

wide  area  network 

Personnel  and  Readiness 

WISSARD 

What  If  Simulation  System 

UTD 

Unit(-level)  Training  Device 

for  Advanced  Research  and 

UTSS 

Universal  Threat  System  for 

Development 

Simulators 

WPC 

Warrior  Preparation  Center 

VIGS 

Video  Interactive  Gunnery 

WWW 

World  Wide  Web 
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